Geoapplications development http://rgeo.wikience.org Higher School of Economics, Moscow, www.cs.hse.ru
2 Agenda • What is geospatial data? • How is it different from other types of data? • Examples of geospatial data • What is geoapplication? • Course goals & organization • Real-world examples: companies using geospatial data • Tentative projects to participate in • Main readings • Quiz • Practical lesson: 3D Globes
3 Geospatial data: 2 types (simplified) RASTER VECTOR matrix What is raster data? http://help.arcgis.com/EN/arcgisdesktop/10.0/help/index.html#//009t00000002000000
4 The main difference from other data types (simplified) RASTER VECTOR matrix each cell has (lat, lon, value) (lat_i, lon_i) for each vertice A fundamental difference: the presence of spatial coordinates
5 Is that all?
6 is there something strange here?? Hint: areas…
7 is Greenland larger than Africa?? Let us consult an Encyclopedia or Wikipedia 2,175,600 km 2 30,370,000 km 2 World size ranking: 14 World size ranking: 2
8 is Greenland larger than Africa?? https://takeitscienceblog.wordpress.com/2016/08/31/how-big-is-your-country/ http://geoffboeing.com/2015/08/map-projections-that-lie/
9 Distances Two airplanes are flying … Plane A: (60 ᵒN, 40ᵒE) → (60 ᵒN, 30ᵒE) Plane B: (40 ᵒN, 40ᵒE) → (40 ᵒN, 30ᵒE) 40ᵒ - 30ᵒ = 10 ᵒ Both traveled the same distance?
10 Everything is not so simple… To name a few: * distances between two points with the same longitudes are different for different latitudes
11 A small caveat: circular/periodic axes longitude range is [ – 180 … +180 ] but what if I want to move 10 degrees eastward from 179 degrees? 179 + 10 = 189 …. actually -171
12 Lesson learned Areas, shapes, angles, distances depend on …
13 Examples of geospatial data: vector
14 Geo prefix → → → → → →
15 Examples of geospatial data: raster & vector More details are at: http://www.wikience.org/tropical-cyclones/wind-swaths/
16 Examples of geospatial data: satellite raster
17 Visualization examples Scankort Denmark Data Extruded Shapes Airspaces NASA World Wind engine We can do more together during course work Video on terrain Analytic Surface Much more examples are at: http://goworldwind.org/demos/
18 Course goals & organization Topics cover technologies Satellites, for a complete geospatial smartphones, GPS data management stack: (1): basic theory: geographic coordinate systems, geocoding, vector and raster data types and operations (3): I/O with vector (SHP, KML, GeoJSON, WKT) and raster (GeoTIFF, NetCDF, HDF) formats (4): network protocols for geodata exchange: WCS, WFS, WMS, OPeNDAP (4): frameworks: Mina, Netty that is used by Twitter (3,4,5): spatial databases and servers: PostGIS, SciDB, GeoServer, TDS (5): software libraries for geodata processing: vector Java Topology Suite and raster ImageMagic, GDAL (6): visualizing geodata with NASA WorldWind, Mapbox, Leaflet, CartoDB
19 Course assessment Code Description Contribution, % LP Lecture presence 20 PP Work during practical lessons 5 CW Control work 10 HA1 Home assignment 1 30 HA2 Home assignment 2 15 EX Exam 20 TT Total contribution
20 Geospatial technology stack (not exhaustive) Visualization Processing Libraries in Java, Python, R, JavaScript, … network protocols Transmission WCS, WFS, WMS, OPeNDAP Almost all DBMS are geo-enabled DBMS Storage Files SHP, KML, GeoJSON, GeoTIFF, NetCDF, HDF
21 Geospatial theory stack (not exhaustive) GPS, WiFi, geohash , … Processing Topology Notations ISO/IEC standard Projections
22 The technologies and theory are building blocks for solving real-world tasks
23 The technologies and theory are building blocks for solving real-world tasks Example task: Because of dramatic traffic increase, the town planners want to widen the main road and add a second lane. The lane width is 20 meters. Find properties that must be demolished. house new lane (its central line) main road tree
24 The solution (DBMS, WKT, Buffer, Topology, SQL, …) DBMS SELECT a_geom, address, owner FROM city_database WHERE ST_Intersects ( ST_Buffer( ST_GeomFromText( " LINESTRING (290 230,…) " , 4326 ), 10), a_geom ) BUFFER ( ST_Buffer ) house POLYGON ((325.5278640450004 258.94427190999915, 326.90755828109235 , ….)) new lane (its central line) main road tree LINESTRING (290 230, 330 250, 390 260, 440 260, 495 252, 535 238, 560 220, 580 190, 590 150) WKT: ISO/IEC standard
25 Consumers of geospatial data Who needs geospatial data? How do they process them? What kinds of geospatial data exist?
26 Consumers of geospatial data: motivating examples http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf
27 Consumers of geospatial data: motivating examples http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf
28 Consumers of geospatial data: motivating examples http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf
29 Consumers of geospatial data: motivating examples http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf
30 Consumers of geospatial data: motivating examples http://www.spatial.cs.umn.edu/Book/sdb-chap1.pdf
31 Real-time ship tracking Geoapps: Marine Traffic http://www.marinetraffic.com http://shipfinder.co/ https://www.vesselfinder.com/
32 Geoapps: FlightRadar www.flightradar24.com
33 EigenRisk Hurricane swaths, flood footprints, assets Spatial operations Visualization Hurricane Katrina, 2005
34 Zillow Selling real estates Filter, nearby, etc.
35 Examples of geoapplications – 2GIS Places Positioning Routes ….
36 Yelp Seek restaurants, hairdressing saloons, etc. + add other
37 UrtheCast Live video streaming from International Space Station A platform for imagery processing
38 Sputnix, Dauria – Skolkovo residents Native satellite manufacturers – ensure Russian national interests Precision agriculture Blooming in Каховском водохранилище Д.К. Мозговой и О.В. Кравец «Использование данных MODIS для экологического мониторинга и контроля чрезвычайных ситуаций»
UAV: Unmanned Aerial Vehicles http://www.youtube.com/watch?v=4E rEBkj_3PY
40 Consumer: Data: wind speed and direction reanalysis and forecasts – raster data Optimize wind farms locations and operation For calendar year 2014, the electricity produced from wind power in the United States amounted to 181.79 terawatt-hours, or 4.44% of all generated electrical energy . https://en.wikipedia.org/wiki/Wind_power_in_the_United_States
41 Wind speed data Energy sector, hazard monitoring, etc. http://wikience.org https://earth.nullschool.net
42 Thermal power stations Optimize load depending on climate/weather forecast -> reduce penalties Air temperature is one of the crucial factors upon which the station load depends
43 Railroads Qingyun (Jeffrey) Xie, High Performance Raster Database Manipulation and Data Processing with Oracle Spatial and Graph
44 More examples
45 Our Projects ChronosDB – a new DBMS R.A. Rodriges Zalipynis. ChronosDB: Distributed, Michael Stonebraker File Based, Geospatial Array DBMS Rio de Janeiro, Brazil 27 – 31 August 2018 VLDB (Very Large Data Bases) – a top conference on Big Data, Distributed Systems, Databases University of California, Berkeley, University of Michigan, 2018: the only talk from Russia Massachusetts Institute of Technology 10 years ago: the previous talk IEEE John von Neumann Medal (2005) from Russia ACM Turing Award (2014) ChronosDB PostgreSQL, Vertica, Streambase, outperforms SciDB by Illustra, VoltDB, SciDB up to 75 × on average https://db-engines.com/en/ranking
46 Moscow & Rio
47 Corcovado The photo is not mine
Tentative course work topics JavaScript: Web GIS (Geographical Information Service) Data compression, processing, visualization, user interaction www.climate.wikience.org
Tentative course work topics performance evaluation DASK GeoTrellis RasDaMan SciDB …. We will develop distributed & multithreaded application to simulate a large load Performance evaluation
Tentative course work topics: machine learning https://habrahabr.ru/company/avito/blog/325632/ https://habrahabr.ru/company/ods/blog/325096/
51 Main readings http://www.spatial.cs.umn.edu/Book/
52 Web sites http://flowingdata.com/
53 Web sites http://geoawesomeness.com/
Recommend
More recommend
