TENS OF THOUSANDS OF EMAILS Bastian Albers https://www.whistleout.com.au/MobilePhones/Guides/Mobile-broadband-usage-guide Or you could make your others really happy and finally write ALL the mails that you are procrastinating … Without attachments of course.
@misprintedtype Our applications should provide the base and work offline per default . The network is the one to enhance our application with new, outside data.
@misprintedtype No matter which technology stack we use… Peer-to-Peer network, direct connection to the server or a USB-Stick , whatever works to transfer the data. It’s not important HOW you transfer the data or application on a device . What is important is, we have saved it all locally and enhance the sync through the network .
the REAL world @misprintedtype Not sure , this is true ? Let’s check out what is happening in the real world and how others solve their offline issues .
the Router @misprintedtype Let’s start with… the Router …
https://ingevald.files.wordpress.com/2009/02/sweden-river-fog-morning-looking-at-sun.jpg @misprintedtype Imagine … A beautiful morning. You’ve decided to work remotely and are sitting in an „ internet cafe “. You’d love to check your mail ? Cool! Connect your laptop to the router . Open your email app and send the request to get your mails . Close the laptop and come back in the evening … wait, WHAT !? Why you might ask ?
https://ingevald.files.wordpress.com/2009/02/sweden-river-fog-morning-looking-at-sun.jpg @misprintedtype Well… There is no „ physical “ connection to the internet like a cable that the internet cafe is using. They just have a router that saves your request and connects via bluetooth to a…
@misprintedtype bus ! It syncs the requests and off it goes. Literally. The bus arrives at a stop after half of the route to the hot spot with physical connection and syncs with another bus. Also exchanges not just the data, but also the passengers and luggage.
@misprintedtype The buses sync via bluetooth previous requests and results . Both buses return . Literally and yeah… you get your emails . Hopefully . If you were able to authorize .
Saami Network Connectivity (SNC - Service) Using NAT zones , DTN routing && DakNet network . @misprintedtype The tech behind the SNC Service is a combination of NAT zones, DTN routing and the DakNet network. (Not to be confused with the DarkNet .)
Learnings (requests & assets) @misprintedtype So, what are our learnings here?
Check your requests ! @misprintedtype Actually check your requests, because beside you should be aware of what your are actually doing… Do you need 20 requests or would actually just 2 be enough ? Please serve uglify and minified files . Do you really need that 5MB background image on mobile ?
beware of Lie-Fi @misprintedtype You should be aware of Lie-Fi , which is the state , where your device checks your connectivity situation, assures that you are online (pinky promise), but there is actually no data passed through.
data queries with bandwidth ? @misprintedtype We could go with checking the bandwidth after the very fist ping right? Yes, that could be a thing ! And we could develop something similar to media queries and device sizes but just with bandwidth . So we can load assets per detected TRUE bandwidth.
@misprintedtype Great idea, right ?! But the bandwidth can differ per request , even per package , especially when you are moving , so e.g. are on a train .
data queries with bandwidth @misprintedtype We would need to ping not just per every request, but constantly. You can’t rely on requests to come back . Please add a timeout to kill your request or make the app work fully independent of requesting data.
enhance assets with network @misprintedtype This is why we need to separate updating cycles of the assets and the data. The update of assets is less important than actually syncing data.
save basic set of UI locally @misprintedtype Save a basic set of your UI assets locally (to go easy on your storage ). Enhance it with e.g. web fonts , images etc. Whatever you need … Offline should be the default state and we need to make the bandwidth assumption by the device we detect .
keep the caches clean @misprintedtype Take responsibility for the caches and keep them clean . What could be better to use to manage your caches, than Service Worker . No intro in Service Worker today . Sorry. Just one thing…
@misprintedtype If you want to use the AppCache for quite a bit, because browsers are dropping the support hard , but also for migration from AppCache to ServiceWorker, please check out the JakeCache .
SVG over IMG @misprintedtype Images… Hard to deal with within the offline context … Saving images in the in-browser storage? Good way to go, but it is still very painful because of storage limitations and very different storage limit implementations . Porting decorative images, like icons and logos to SVG and CSS is a great way to go to cut off a bit of the overhead .
base64 over IMG @misprintedtype Encoding assets in base64 and storing the String in the in - browser storage of your choice could be a way to go. For example, saving fonts like that and loading them straight from the storage, improves the performance a lot. This is even possible for images, like a logo . But please, just do not do this for all of your images. Don’t build the town, build the house. Remember ? The strings are huge as we all already know. And in-browser storages are is limited .
base64 is expensive && heavy @misprintedtype Also decoding base64 on the fly is a heavy task. So it slows down your site. Especially in combination with DOM - manipulating frameworks like React. You’ll also find yourself running into race conditions here. -> drink Let’s check out the next use case… It is a bit heavier, as already mentioned. Just to let you know in advance .
the Pandemic @misprintedtype -> the Pandemic
@misprintedtype Twenty-fourteen and twenty-fifteen , the ebola pandemic in parts of West Africa was a really scary issue.
@misprintedtype I was involved in a project by the NGO eHealth Africa that build two applications to help stop the pandemic . The first application to solve one of the issues was… providing information for the population, because information is the key to prevent a pandemic to spread or kill .
@misprintedtype The process was… The population was informed about the number they could call to ask about ebola symptoms , to report someone sick or even dead. The call center tracked the calls with the application we’ve build, collected all the provided information from the caller and sent out a team , if needed.
@misprintedtype They added information through the application like the name, location and additional information. Also added the phone number of the caller , so they could call back for any further questions. Data analysis and filtering was really easy thanks to the application .
rm -rf ~/ @misprintedtype They fought with issues like unexpected and constant power loss . The generators jumped in after some time, but we could not afford to lose data. Just one case could have caused another outbreak and many deaths .
first learning (data) @misprintedtype Our first learning here was …
Decouple server && client @misprintedtype Decouple server and client, to serve the needs of a eventual contingency.
build a task / message system @misprintedtype Build in a task and message system, that can act independently from the connectivity situation . For example… for data CouchDB, especially in combination with PouchDB proved their usability in the real world really, really well .
@misprintedtype CouchDB a NoSQL database that replicates , which is exactly what we want for sync. CouchDB comes with a suite of features , such as on-the-fly document transformation and real-time change notifications , we need for our message and task system, but also for a decoupled architecture.
@misprintedtype PouchDB is an open-source JavaScript database inspired by CouchDB that is designed to run well within the browser and store all your data locally while offline.
@misprintedtype It sync’s with CouchDB thanks to the CouchDB replication protocol, when the application is back online .
save data documents locally @misprintedtype With PouchDB we actually have documents which are stored locally . So we won’t not flush the caches during by accident.
prevent merge conflicts @misprintedtype One of the biggest issues with syncing data are merge conflicts . Because PouchDB and CouchDB speak the same protocol and sync automatically when we are back online and we do not need to take care of it .
@misprintedtype The second application we’ve build, was to actually track patient data in the field. After examining the infected patients, who had to stay in quarantine for 21 days. The team added the data to a general database through the application plus got lab results from several different entry points and very different file types .
@misprintedtype The data was imported , saved in a global database and synced all across all kind of devices. This all sounds super messy, right? What do you do with a mess ? Well, I am a parent . So I am used to cleaning up a mess …
second learning (schema) @misprintedtype Our second learning here was …
define flexible data schema @misprintedtype We added an organization system. An interface to map the data in one defined schema. The schema was flexible enough to be extended , if needed… Thanks to the NoSQL database behind it.
test data schema constantly @misprintedtype Data was imported automatically when it arrived (mostly per mail ) or we added data on the fly . Constant automated testing and a high test coverage for the importer and the interface was a must.
prevent merge conflicts @misprintedtype The schema and the testing helped us a lot here to prevent merge conflicts before the get go . CouchDB and PouchDB helped us to take care of the sync, merge and flexibility .
prevent merge conflicts @misprintedtype Big bonus also? The schema also helped us with making filter and analyze the data, so visualizing it was really easy.
10/2014 12/2015 @misprintedtype We helped to curtail the pandemic , because of clean data structure and constant testing of the applications. If anyone tells you that those things don’t matter or you don’t have any time for those, you now know what to reply . Almost lunch … Guess, well picked theme for the next use case. Sorry in advance…
the Pizza @misprintedtype -> the Pizza
@misprintedtype We all love to order food, right? Did you ever check the localStorage when ordering ? Just to see , if they save the cart items in it? A friend of mine did this once and went bit further out of curiosity . He was wondering if they are actually validating their checkout on the server .
@misprintedtype So he manipulated the checkout and subtracted one cent, send it out and waited . The pizza came. And no, there was no server side validation .
Learnings (security) @misprintedtype What do we learn here?
validate on server @misprintedtype In contrast to the limited possibilities of our use case before , this company could have validated the data on the server .
validate on server @misprintedtype But this is sometimes not possible , especially when everything needs to work without any requests . Security is still a very new topic to offline first and not many tools are provided yet. And I’d be happy to chat especially about that topic after my talk .
CORS <3 @misprintedtype Although many of us seem to hate CORS sometimes, it provides already a great set of rules to make web applications more safe . But it does not solve the issues with the in- browser storage manipulation .
encrypt local data @misprintedtype One way to go would be to encrypt your data . When you use PouchDB , you can add CryptoPouch , which encrypts your Pouch documents . Big downside here is, the document get’s encrypted , but not the ID . Which can lead back to security issues . But at least you can’t change the data anymore in the console .
More recommend
