FOSDEM 2020 Online: wrestling Copy / Cut / Paste to usability Collabora Productjvity Michael Meeks <michael.meeks@collabora.com> General Manager at Collabora Productjvity @michael_meeks, mmeeks #libreoffjce-dev irc.freenode.net “Stand at the crossroads and look; ask for the ancient paths, ask where the good way is, and walk in it, and you will fjnd rest for your souls...” - Jeremiah 6:16
C Overview Web copy/paste technologies Baroque and paranoid ● Riddled with inelegant back-compatjbility ● Layers of un-necessary and unhelpful ‘security’ features. ● Comparison / before and afuer How we improved things. collabora online . com 2019-09 .. 2
Beforehand: the Competjtjon & the Problem
C Chrome / GDocs spreadsheet copy/paste collabora online . com 2019-09 .. 4
C Offjce.com / Excel copy / paste ... collabora online . com 2019-09 .. 5
C Online – old-style copy/paste: text → CSV import dlg. collabora online . com 2019-09 .. 6
C High fjdelity ‘internal’ shortcut ... A large proportjon of copy/paste is for ● shuffming / pruning bits of the current document. So – initjal context-menu / copy-paste ● short-circuited internally Doing all the work inside the Online ● server. Partjcularly important for eg. large ● spreadsheet cell area movement. This requires formula re-writjng. ● Leaving plain-text for external use. ● collabora online . com 2019-09 .. 7
Web Clipboard technologies.
C Web copy/paste is baroque & paranoid beyond belief. APIs for copy/paste are very mixed In many browsers copy/cut can put ● ~anything on the clipboard ~whenever they like. But they have to work-around layers ● of obsolete ‘security’ foo. 1. Find/focus/select content-editable ● 2. document.execCommand(‘copy’) ● 3. catch events if you get them ● ● 4. tweak your content- editable, ● 5. retry from 1. collabora online . com 2019-09 .. 9
C IE11 – had a great initjal approach: collabora online . com 2019-09 .. 10
C Google Chrome / Docs – joined up thinking ? Google Chrome’s ● Security team hardened by many issues → so rightly paranoid. Google Doc’s ● Users cannot copy/paste using context menus → make a plugin for that. The Security situatjon ? ● Snooping is indeed bad: ● IE11 solutjon was a good one. ● Signed sites you trust … ? collabora online . com 2019-09 .. 11
C Other web security (mis-)features. Piles of Baroque nonsense More Piles of Baroque nonsense IE11 All schemes revolve around a hidden / ● ● transparent / ofg-screen / textarea / Can only do HTML copy/paste out of a ● contenteditable. hidden content-editable You have to fjll / empty the HTML out ● JQuery → a great idea for cross-platgorm ● of that manually in a very odd support. sequence. iOS: JQuery foo’s click’ isn’t. ● Secure Copy/paste types: ● It is a wrapped touch event ● text/plain, text/html, text/rtg ● Which doesn’t have the popup ● Image/* ● security context we need. but in reality unpacked ● Breaks only iOS ● platgorm Bitmap ● needs <a href=”#”> links Gettjng meta-data through: text/html ... ● collabora online . com 2019-09 .. 12
C Mobile Web clipboard problems ... Luckily we can detect if we succeeded ? Pressing Ctrl-C is harder … Not trivially: ● iOS has a nice butuon on the keyboard ● return values from execCommand is ● Android does not ● unreliable. But GBoard does ● So set incrementjng global serial of ● successful c/c/p operatjons But GBoard converts ● iOS HTML→Text and includes ● random <head> tags. Amazing … if security context is not ● perfect: everything appears to succeed. GBoard creates text input ● You just successfully push data to a behavioural horrors of its ● clipboard no-one else can see (not even own. you). collabora online . com 2019-09 .. 13
C Fundamentals: Synchronous & no negotjatjon All data at once ... Synchronous Web clipboard APIs All (sensible) OS APIs look like ● Are all synchronous ● “tell me what formats you support” ● They all assume that all the data ● “later (if someone pastes) – I’ll get ● you want to copy/paste is already back to you to ask for the (few) they in the browser. actually want. Horrible for us, VDI clients, and ● Why ? 10^9+ cells in a spreadsheet. ● other large / server hosted Not the web: ● applicatjons All formats must be inserted ● When we get Ctrl-X ● synchronously Need all the data-ready to put to ● the OS clipboard. Consider paste-as PNG of spreadsheet ranges collabora online . com 2019-09 .. 14
Improving things ...
C Users: KISS – for the simple case … and then ... Plain text Try to explain the sorry situatjon to users: As you select ‘simple’ selectjons – these are ● Necessary to click-again afuer ● pushed KIT → browser as HTML download to copy / cut → for security. Paste to another Online instance / window ... Shortcut, no need to download via a custom ● meta-origin. Complex cases Push an interestjng text to the clipboard: ● “To paste outside Online, please fjrst ● click the 'download' butuon” collabora online . com 2019-09 .. 16
C HTML with meta-fjle bits ... "<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> When we get paste data <html> <head> If it has ‘our’ meta then: <title>Stub HTML Message</title> ● <meta http-equiv="content-type" content="text/html; If that is for the same ● charset=utf-8"/> document-id, part, <meta name="origin" content="%2Flool%2Fclipboard%3FWOPISrc %3Dfile%253A%252F%252F%252Fopt%252Flibreoffice%252Fonline view etc. %252Ftest%252Fdata%252Fhello-world.ods%26ServerId Short-circuit to ● %3Dc2bde695%26ViewId%3D0%26Tag%3D6da096542b9602ad"/> uno:Paste </head> <body lang="en_US" dir="ltr"> Job done ● <p>To paste outside Collabora Online, (as before) please first click the 'download' button</p> Otherwise download </body> ● </html>" the data collabora online . com 2019-09 .. 17
C Download & re-up-load to paste Paste of ‘our’ data: Infrastructure to make this work New web / clipboard / end-point Async Download source data ● ● Connects to the relevant Kit process to (as all mime types) ● serve & set the data. Async Up-load data ● Kills fjnal need for ‘X’ libraries → safer ... ● (as all mime types) set it to Kit clipboard. Gotchas Send an .uno:Paste to Kit. ● Dialog paste / paste-keys ● Initjally empty Kit clipboard → no ‘Paste’ ● ‘Download’ of data: Closing a Document → loose its clipboard ? ● Async Download of (just) HTML ● New synchronous Kit shutdown ● Base64 embedded images. state machine… urk. ● collabora online . com 2019-09 .. 18
C Afuerwards ... collabora online . com 2019-09 .. 19
C Afuerwards + explicitly copying charts: collabora online . com 2019-09 .. 20
C Other miscellaneous details …. Security Mac / Safari We push a new clipboard access key every 2 ● Pastjng from Safari – we get RTF ! ● minutes Lovely, powerful, rich-text … ● We accept this + previous key ● Unfortunately – doesn’t contain ● → between 2 & 4 minutes to ● images. copy/paste. Unfortunately – by design can’t tell Or you can disable copy/paste with the ● ● relevant WOPI-like property who is sending you the data… So don’t accept RTF on the Mac ● Text/plain – regression ... Web can’t easily convert HTML → text !? ● Will need to send text/plain too. ● collabora online . com 2019-09 .. 21
Conclusions Collabora Productivity Our codebase has a fantastjc heritage & rich copy-paste code ● But connectjng it was an epic – primarily fjghtjng web api ● Online can now copy rich types – leading in its class. ● Thanks to: Ashod, Szymon, Marco, Miklos, Aron and others… ● Thanks to all of our customers & partners who make this possible. ● Questjons ? ● Oh, that my words were recorded, that they were writuen on a scroll, that they were inscribed with an iron tool on lead, or engraved in rock for ever! I know that my Redeemer lives, and that in the end he will stand upon the earth. And though this body has been destroyed yet in my fmesh I will see God, I myself will see him, with my own eyes - I and not another. How my heart yearns within me. - Job 19: 23-27
Recommend
More recommend
