Upgrade to MongoDB 4.0 Antonios Giannopoulos DBA @ - PowerPoint PPT Presentation

Upgrade to MongoDB 4.0 Antonios Giannopoulos DBA @ Rackspace/ObjectRocket linkedin.com/in/antonis/ 1

Introduction Antonios Giannopoulos www.objectrocket.com 2

Overview • Upgrade Procedure • Application Layer • Middleware • Database Layer • Rollback Procedure • Why 4.0? www.objectrocket.com 3

MongoDB 4.0 MongoDB 4.0 released on June 26,2018 Current minor version is 4.0.3 You can obtain it from: - Mongo Inc Download Center - Percona Server Download Center - Repos like yum, apt-get… www.objectrocket.com 4

MongoDB 4.0 www.objectrocket.com 5

Upgrade Replica-Set Upgrade the secondary , one at a time Shut down the mongod instance o Replace the binary with the 4.0 binary o Restart the member o Connect a mongo shell to the primary Issue rs.stepDown() o Ensure a new Primary is elected o Upgrade the ex-Primary Shut down the mongod instance o Replace the binary with the 4.0 binary o Restart the member o Connect a mongo shell to the primary Enable backwards-incompatible 4.0 features db.adminCommand( { setFeatureCompatibilityVersion: "4.0" } www.objectrocket.com 6

Sharded Cluster Stop Balancer sh.stopBalancer() o sh.getBalancerState() o Upgrade config servers Use the replica-set steps o Upgrade the shards Use the replica-set steps o Upgrade the mongos One at a time o Replace binary and restart o Enable backwards-incompatible 4.0 features db.adminCommand( { setFeatureCompatibilityVersion: "4.0" } ) s1 s2 Restart the mongos www.objectrocket.com 7

Application • Upgrade Driver • Upgrade Language • Inspect the code base www.objectrocket.com 8

Driver Your driver must be compactible with 4.0 www.objectrocket.com 9

Driver www.objectrocket.com 10

Driver www.objectrocket.com 11

So just upgrade the driver? The underling language version must also be compactible with your driver Mongo 4.0 Compatible www.objectrocket.com 12

Upgrade the driver/language Perform safe upgrades /always have rollback in mind Python supports “ virtual environments ” Python “virtualenv” creates an isolated environment for Python projects. Each project can have its own dependencies, regardless of what dependencies every other project has. There are no limits to the number of environments ,they’re just directories containing scripts www.objectrocket.com 13

Upgrade the driver/language www.objectrocket.com 14

Upgrade the driver/language What about other languages: There are equivalents of “virtualenv” like JAVA use classpath Ruby use Ruby Version Manager (RVM) www.objectrocket.com 15

Upgrade the driver/language Do not forget frameworks and their dependencies o Very popular in PHP o They have their own compatibility matrix Should I always use the latest driver? o Check the change log o Check the open bugs www.objectrocket.com 16

Driver version? Dunno Inspect the logs Manipulate the logs to export driver name & version Quick hack: less /var/log/messages |grep "driver: { name" | awk '{print $16, $17, $18, $19}' | uniq www.objectrocket.com 17

Driver version? Dunno Inspect the system.profile Collect a representative sample On a sharded cluster, all shards must examined. www.objectrocket.com 18

Inspect the codebase Each version Removes & Deprecates operators. Remove : Operator is no longer available Deprecate: Operator will be removed to the next version Take Actions: o Patched the codebase against the removed operators o Plan to replace the deprecated operators www.objectrocket.com 19

Inspect the codebase Removed Operators: o $isolated operator: If you have an existing partial index that includes the $isolated operator or a view that includes a $isolated operator, recreate the index or view without the operator in the definition before upgrading. Deprecated Operators: o maxScan o geoNear command o copydb and the clone commands www.objectrocket.com 20

Inspect the codebase Exception handling o Error descriptions may change between versions o Error format may also change o New error codes may introduced o New warning codes may introduced www.objectrocket.com 21

Middleware • Monitoring • Backups • Deployments • Utilities • OS changes • TLS/SSL www.objectrocket.com 22

Monitoring The monitoring system must be able to connect to MongoDB Vast majority relies on db.serverStatus() A newer version may be necessary to installed prior to upgrade www.objectrocket.com 23

Deployment & Managment Automatic deployment scripts also connect to MongoDB For example: o Deploy a replica-set (rs.add(), rs.status()) o Add a shard (sh.addShard()) A change to error code formats may affect deployment scripts User-roles may also change Management tools may also affected www.objectrocket.com 24

Backup - Filesystems Snapshots - Copy files - Hot Backup (Percona Server) - Mongodump They should all work but testing is recommended. In the case of restore, a downgrade may be required. www.objectrocket.com 25

OS Removes support for: o SLES 11 o Ubuntu 12.04 o Debian 7 Deprecates: o Windows 7/2008R2 o Windows 8/2012 o Windows 8.1/2012R2 o Ubuntu 14.04 Compatibility matrix: https://bit.ly/2ys8Crs www.objectrocket.com 26

TLS Removes support for TLS 1.0 o On systems where TLS 1.1+ is available Latest PCI compliance standards require the use of TLS 1.1+ If you need to support TLS 1.0 (hopefully temporarily): o Set to none either net.ssl.disabledProtocols or --sslDisabledProtocols Test: openssl s_client -connect <host>:<port> -tls1 (tls1_1 and tls1_2) www.objectrocket.com 27

Database Layer • Configuration files • Prerequisites • Deprecated items • Storage engine • Miscellaneous • FCV www.objectrocket.com 28

mongo.conf net.transportLayer cant be set to legacy anymore net.host: Resolves localhost IP address as configured instead of assuming 127.0.0.1 Can’t combine storage.journal.enabled: false with WiredTiger storage engine Can’t combine storage.indexBuildRetry with replication.replSetName When ssl.allowInvalidCertificates: true with x.509 authentication, an invalid certificate is only sufficient to establish a TLS/SSL connection but is insufficient for authentication . www.objectrocket.com 29

Prerequisites In order to upgrade featureCompatibilityVersion must be set to 3.6 How to check: o db.adminCommand( { getParameter: 1, featureCompatibilityVersion: 1 } ) How to set FCV to 3.6: o db.adminCommand( { setFeatureCompatibilityVersion: <version> } ) , where <version> ”3.6” It automatically enables 3.6 new features www.objectrocket.com 30

Prerequisites 3.6 new features UUID for collections o $jsonSchema document validation o Change Streams o Chunk aware secondaries o View definitions, document validators, and partial index filters that use 3.6 query features o Sessions and retryable writes o Users and roles with authenticationRestrictions o Sessions: Creates a system.session collection o On sharded clusters is sharded on {_id:1} o Has a 30 minute TTL index o Pre-3.6.7 bug, wasn’t creating TTL and sharding o May trigger a performance overhead (updates/deletes) o www.objectrocket.com 31

Prerequisites In order to upgrade pv must be set to pv1 How to check: o pv=rs.conf().protocolVersion How to set pv1 : o Login to every replica-set and execute cfg = rs.conf(); cfg.protocolVersion=1; rs.reconfig(cfg); Enjoy the benefits of pv1 www.objectrocket.com 32

Prerequisites Master-slave replication , no longer supported You must upgrade to a replica set Prior the upgrade Conversion involves downtime & affects high availability o Stop mongo o Start Master with --replSet <setname> o Initialize the replica set rs.initiate( { _id: "<setname>", members: [ { _id: 0, host: "<host:port>" } ] } ) o Add nodes, rs.add("<host:port>" ) www.objectrocket.com 33

Prerequisites Removes MongoDB Challenge-Response (MONGODB-CR) auth mechanism Deprecated since 3.0 version. Only possible if you upgraded from 2.6 How to check: db.getSiblingDB('admin').system.users.find({"credentials.MONGODB-CR":{$exists:true}}) How to upgrade to SCRAM: db.adminCommand({authSchemaUpgrade: 1}); Considerations o authMechanism on connection string o Local users on sharded clusters www.objectrocket.com 34

Storage engine MongoDB 4.0 deprecates MMAPv1 It’s likely the next version to support only WiredTiger You should consider a switch to WiredTiger Change to WiredTiger may be challenging: o In-place Updates o Range Queries o Different HW specs o Different shard keys www.objectrocket.com 35

Storage engine storage : storage : dbPath: <data dir> dbPath: <data dir> engine: mmapv1 engine: wiredTiger mmapv1: mmapv1: <mmap configuration> <mmap configuration> wiredTiger : collectionConfig: blockCompressor: <value> engineConfig: cacheSizeGB : <value> directoryForIndexes: true journalCompressor: <value> indexConfig: prefixCompression: <value> www.objectrocket.com 36