filter_list

Using system clipboard from GWT

Tuesday 30.10.2012

The problem

There is no official support in the HTML standard to access the system clipboard. With that being said, there are a few tricks that can be utilized to work around this limitation. Most browsers support copy and paste operations only from text boxes or other editable fields. The content of these fields can be transferred to and from the system clipboard using the right-click menu or keyboard shortcuts.

The system clipboard will contain one or several versions of its content. The content could have a format of for example text, image data, html, xml and so on. Whenever the source application determines that a copied element could be used in different formats, multiple copies of the selected element in various formats will be placed on the clipboard. The application that reads the clipboard content could then fetch the content in the format it wants.

The exception

Internet Explorer is the only browser that support programmatic access to the system clipboard. When you run this operation in your code, you are greeted by one or two dialogs asking you as a user if it is OK for you that the running application should be allowed to access the system clipboard. When you have allowed clipboard access for this application, you will not be asked again until you close the browser window. You could of course add the website in question to your list of trusted sites, and set the security level for trusted sites to low. Or even change the specific setting. You would then open yourself to a world of security holes, which makes this solution less than recommended. To make matters even worse, IE version 6 and below will not even ask before accessing the system clipboard, opening for even more security holes.

Google Docs

Lets start by looking at how Google solved this for Google Docs. In the article "Google Docs and Clipboard Access", some of the issues found while implementing support for copy and paste for Google Docs are described. When writing a document in Google Docs (or Blogger), you are writing the text within a large editable area. This area allows keyboard shortcuts because the browser natively support copy and paste operations using keyboard shortcuts within a text area or text field handled by the browser. Using the browser context menu to do copy and paste will also work in such a field. A customized right-click menu or a menu bar will not be able to do these operations for most browsers.

So how does Google Docs handle menu based clipboard access from the different types of browsers? IE gets programmatic access after showing one or two warning dialogs. Firefox gets a dialog that asks the user to please use keyboard shortcuts. Chrome gets a message telling the user to install their Chrome specific Google Drive application. When a company as large as Google is not able to solve this problem, the problem has to be quite complex.

Some possible solutions

These problems show that we cannot rely on stock JavaScript alone to access the system clipboard. What assisted alternatives do we have to solve this problem?

Internet Explorer clipboardData

As mentioned earlier, IE allows you to access the system clipboard directly. Even though you are greeted with some annoying warning dialogs when your application tries to access these API's, this approach gives full access to the system clipboard. This is realized using the IE version of the clipboardData object.

Data is written to the clipboard in this way:


var success = window.clipboardData.setData("Text", "The text");

Data could then be read using something like this:


var data = window.clipboardData.getData("Text");

This approach would work if you are targeting IE alone, or if you want to create a separate implementation for each browser. We want an implementation that supports several browsers, preferably using a similar method for all major browsers.

Adobe Flash

There are several libraries for accessing the clipboard using Flash. For the sake of an example, I will be using one called Zero Clipboard. When using a Flash helper you should be aware that Flash imposes a few restrictions on your use of the clipboard. Because of how the Flash sandbox is implemented you can only push data, while pulling is not allowed. Using Flash Player version 9 or lower you could push data at any time. From Flash Player version 10 an additional restriction were added. Here you can only paste data when the user does a user interaction within the Flash movie, like a mouse click or a button press.

Data can be pushed to the system clipboard using something like this:


var clip = new ZeroClipboard.Client();
clip.setText("Copy me!");

As this solution is a one-way street, it might not solve your problem. Because of how the sandbox were further closed for the version 10 release, these added restriction might also be a hinder for your usecase. One might trick the Flash runtime by setting focus to the element when the user initiates a keystroke or a mouse click. Zero Clipboard solves this problem by placing the movie clip on top of the element in question. These methods are simply bad workarounds, and should only be considered as a last resort.

An another thing to consider is that Flash support is being dropped in the Apple land after their hate campaign against the use of Flash. Adobe tries to defend their position, but now that Flash support is also dropped from Android devices, we can safely say that we should leave this platform behind us.

Microsoft Silverlight

Support for accessing the system clipboard was added in Silverlight version 4. The Clipboard class lets you read and write to the clipboard. Version 5 does however add restrictions similar to the Flash plugin, in that these operations must be the result of user interaction. As long as these methods are called as the result of a keyboard press event or a mouse press event, access to these functions are allowed. It should also be noted that the Windows Phone version of Silverlight only allows you to set the content of the system clipboard. It should also be noted that Microsoft themselves now seems to abandon Silverlight. It is therefore preferable to not rely on Silverlight for clipboard access.

Java applet

Pure Java applications is able to access the native clipboard perfectly. This can be seen to its full effect in full-blown Java-driven applications like Netbeans and Eclipse. Applets used to work in the same way. When Oracle released Java Plugin 1.6.0_24 in February 2011, they closed access to the system clipboard as a way to stop a security hole. What this means in practice, is that you can copy and paste data between Java applets as you would expect them to. The data you copy does however never reach the system clipboard. The Java runtime uses an internal clipboard that will only feed applets with data.

This restriction can be overridden in several ways. The simplest way is to modify your local policy file so that applets have access to the system clipboard. This requires that your end users are relatively tech-savvy, and that they will accept to do a change like this, knowing that it will also affect how applets are handled on other web pages. This is highly unwanted, and one would really like a better solution.

An another way to override the clipboard sandbox is to use a verified certificate. Such a certificate can be bought from sources like Verisign / Symantec. These certificates cost as of today $499. These certificates open the sandbox completely and lets you access everything a desktop Java application could.

With these quirks out of the way, we remain with the problem of communicating with the applet from JavaScript. Communicating with JavaScript from a Java applet is relatively easy, but problems start to occur when we want the communication to be initiated from JavaScript.

The first version of JavaScript initiated communication was called LiveConnect. This technique was implemented as part of the Netscape 4 browser in June 1997. This communication still works in Firefox and Opera, but Chrome has dropped support, while IE has never supported it.

LiveConnect has been surpassed by NPRuntime. This method is supported by all major browser with the exception of IE. On their own pages the Chrome developers does not recommend the use of NPAPI plugins, as they can not easily be fit within a sandbox, and is therefore a security threat.

With all these problems we can safely say that a Java applet is not a recommended solution. With problems like sandbox restrictions and communication problems, as well as browser incompatibilities, it will be really hard to make a solution that works in all situations. It should also be noted that the startup of a Java applet requires a few seconds the first time. The next time you visit a page with the same applet, it start up immediately. This startup delay will be repeated when you close the browser.

execCommand

The execCommand method lets you execute internal functionality of the browser. These calls could for example be used to create a rich-text editor.

For copying the text, something like this would suffice:


function copy() {
  var area = document.getElementById("area1");
  area.focus();
  area.select();
  document.execCommand("copy", false, null);
}

For pasting the content back into the browser, this method could be run:


function paste() {
  var area = document.getElementById("area2");
  area.focus();
  area.select();
  document.execCommand("paste", false, null);
}

Note that the two last parameters are required in Firefox and Opera, while they are optional in IE, Chrome and Safari.

The result and availability of these calls vary widely between browsers. For our use we need the "copy" and "paste" commands. These two commands are available in IE and Safari only, while the other commands are available for most browsers. When running these commands on these browsers the selected text will be copied to the clipboard for the "copy" command, while the "paste" command insert the clipboard content within the component that currently has focus.

Because these commands works in only a few browsers, they aren't too useful for our use. For the browsers that this method works with, the previously mentioned methods that do not require an additional text box is preferred.

Mozilla XUL

Firefox has a language for building user interfaces called XUL. Applications built using XUL can be compiled to native applications, creating truly cross-plattform applications. Within XUL there are functionality to access the system clipboard. Accessing this data could be a lot of pain though.

Accessing the clipboard using XUL in the browser is not available by default. There are four scenarios where accessing the system clipboard is possible:

  • Copy the web pages to your local machine and run it from there, which hardly can be called a modern web.
  • Change some obscure setting within the browser, which in turn excludes the lesser tech-savvy endusers.
  • Activate the security setting using JavaScript. This makes a dialog pop up in the browser asking the user for permission to do so.
  • Digitally sign the JavaScript.

With these options covered, it should also be mentioned that XUL will only work in Firefox. If you want a truly cross-platform solution, this approach will not help you.

Clipboard events

There are six events available for clipboard events. One before and one after event of each of the "copy", "cut" and "paste" operations. How these events work between the different browsers vary wildly. There is also a bug registered for WebKit, which makes it impossible to write to the clipboard events using Chrome and Safari. The official standard also mention security risks with these events, which might be the reason why this bug has been lingering since 2008.

Because we want these operations to work both ways, we need to find an another approach that works both ways in all browsers.

Hidden textarea

An another method is to use a hidden textarea trick. This involves positioning a textarea element outside of the view of the user. When the user then presses any of the keyboard shortcuts for clipboard operations, this textarea will get focus and be selected. The browser will then either copy the text out of this field into the system clipboard or place new text within it. Either way we trick the browser into using the content of this hidden textarea for all copy and paste operations. This is the same trick that is used within CkEditor and TinyMCE rich-text editors.

One might think that it would be easier to hide the textarea within the visible browser area of the screen using display none or similar. This approach would stop browsers from pasting to the textarea, as the browser sandbox will prevent you from pasting to hidden fields. Positioning the textarea outside of the view will also in practice hide the field from view, but this fact is overlooked by the browser.

This approach seems to work fine in all browsers. Although a bit cheaty, this method will allow you to copy and paste text using the keyboard shortcuts.

The implementation

When looking at the various alternatives, I decided to try to implement the hidden textarea trick for the Motify application. The other methods works only for certain browsers, while some only works in very special conditions. This method seems to solve the problem in a somewhat elegant way that is totally transparent to the end user. We've implemented the application using Google's GWT framework, and I've therefore implemented this functionality using GWT concepts.

Step one is to add a textbox hidden out of view. I started this implemention using a text field, but this component do not preserve line changes. All lines were simply merged into one line without newlines. This problem does not occur when using a textarea.


this.textBox = new TextArea();
this.textBox.getElement().getStyle().setPosition(Position.ABSOLUTE);
this.textBox.getElement().getStyle().setZIndex(100);
this.textBox.getElement().getStyle().setLeft(-1000, Unit.PX);
RootPanel.get().add(this.textBox);

Events need to be processed before all other event handlers. We therefore attach a native preview handler.


Event.addNativePreviewHandler(this);

When handling the native events, we restrict event handling to keyboard down events.


@Override
public final void onPreviewNativeEvent(final NativePreviewEvent event) {
    final NativeEvent nativeEvent = event.getNativeEvent();
    switch (Event.getTypeInt(nativeEvent.getType())) {
    case Event.ONKEYDOWN:
        this.onKeyDown(nativeEvent);
        break;

    default:
        break;
    }
}

Key events are then restricted to copy and paste shortcuts:


private void onKeyDown(final NativeEvent event) {
    if (!event.getCtrlKey() || !this.containerRenderer.hasFocus()) {
        return;
    }

    if (event.getKeyCode() == ExtendedKeyCodes.KEY_C
            || event.getKeyCode() == ExtendedKeyCodes.KEY_X) {
        this.textBox.setText(this.getContent());
        this.textBox.setFocus(true);
        this.textBox.selectAll();

        Scheduler.get().scheduleFixedDelay(new RepeatingCommand() {
            @Override
            public boolean execute() {
                TextBoxClipboardHandler.this.containerRenderer.setFocus();
                return false;
            }
        }, INTERCEPT_DELAY);
    }

    if (event.getKeyCode() == ExtendedKeyCodes.KEY_V) {
        this.textBox.setText("");
        this.textBox.setFocus(true);
        this.lastInterceptedPaste = System.currentTimeMillis();
    }
}

We start by checking if the ctrl was used, as we want to check only key events with this modifier. The containerRenderer variable holds a reference to the component where we want to handle copy and paste keyboard events. In our case this is the area used for drawing diagrams. If this component does not hold the current focus, we will not process keyboard events.

Copy and cut keyboard shortcuts (C and X) starts by fetching the content to be placed on the system clipboard. This text is then pushed to the text box, as well as receiving the current focus. The full content of this text box is then selected, ready to be placed on the system clipboard. At this point the textual content assigned to the field will be fetched by the browser and placed on the user clipboard. The final step is to assign focus back to the drawing area. This needs to be done after the browser has moved the content from the text box over to the system clipboard. The delay chosen for my implementation was 100ms. This delay seems to be enough for all browsers to do their needed job. The difference between copy and cut operations are handled outside of this class, where a key down handler removes drawing content when using the keyboard shortcut for cut.

The paste keyboard shortcut (V) starts by removing the current content of the text box. Focus is then assigned to the field, and we remember when we last started a paste interception. The rest of the paste handling is handled outside of the class, which then returns to the paste method within this class.


@Override
public final void paste() {
    if (Math.abs(System.currentTimeMillis()
            - this.lastInterceptedPaste) < INTERCEPT_LIMIT) {
        Scheduler.get().scheduleFixedDelay(new RepeatingCommand() {
            @Override
            public boolean execute() {
                TextBoxClipboardHandler.this.setContent(
                        TextBoxClipboardHandler.this.textBox.getText());
                TextBoxClipboardHandler.this.containerRenderer.setFocus();
                return false;
            }
        }, INTERCEPT_DELAY);
    } else {
        this.internalClipboardHandler.paste();
    }
}

We start paste processing by checking if the time difference between the current time and the last interception is less than the interception limit. This limit is also set to 100ms. The reason why we check for this, is that we want to set focus back to the drawing area only if the paste event was initiated using a keyboard shortcut when focus is assigned to the drawing area. Pressing the copy and paste functions from the toolbar should not change the focus, but instead perform an internal paste operation. When using the keyboard shortcut, the text box will contain content that we then will process and insert into the drawing. The drawing container will then receive the focus.

The paste box as positioned within the Motify application

When we look at the application as it is visualized using the tilt function of Firefox, we can see that the text area is placed to the far left. In this image the text area is currently selected, and is therefore shown in blue on the far left. As we can see, the rest of the application on the right side is completely unobfuscated by the text area.

This solution is currently in place for the Motify application, and works completely transparent for the end user. A normal user expects web applications to behave in the exact same way as desktop applications, and finds it a hassle if the application does not support rudimentary functionality like copy and paste in the way that they expect. We did support internal copy and paste within the same drawing earlier, but with this implementation in place we can now easily copy elements between drawings.

First published at:
http://infposs.blogspot.no/2012/10/using-system-clipboard-from-gwt.html

Motify examples

Wednesday 03.10.2012

As we approach the release of the Motify library we want to show you some of what we have been working on. These examples give a small look into how the final product will be like. These examples are not yet in its final form, and might therefore contain bugs and lack some features. When we release the final version of the library, these examples will be fully polished and contain all the functionality they should have.

Complete example

This example show what a developer can do using the GWT version of the library. This example uses the default GWT interface. While this interface might be good enough for most uses, the interface is completely customizable from code. http://www.motigon.com/demo/

Example browser

The example browser contains several demos that shows how to use various elements of the library. All these examples contain full source code so that it's easy to find an example for most scenarios. The browser will be extended as more features are added to the library. http://www.motigon.no/examples/

JavaDoc

Complete documentation for the Java / GWT library. Here you will find how the library is organized as well as how each method is intended to be called. Details on how parts of the library works with each other can be found in the example browser. http://www.motigon.no/javadoc/

JsDoc

Complete documentation for the JavaScript library. This documentation tries to follow the format of the documentation generated by the JsDoc project. Because this documentation is generated from Java sources that has been exposed as JavaScript, parts of this documentation will not appear quite like its pure JavaScript counterpart. http://www.motigon.no/jsdoc/

2017

2016

2014

2013

2012

2011

2010

2009

2007

2005

2004

2003

2002

2001