INFORMATION TECHNOLOGY TRACK Russell Brown Chapter 13 Standing - PowerPoint PPT Presentation

NACTT STAFF SYMPOSIUM SERIES INFORMATION TECHNOLOGY TRACK Russell Brown Chapter 13 Standing Trustee - Phoenix, AZ Allan Reininger System Manager - San Antonio, TX Chapter 13 Standing Trustee - Mary Viegelahn Carl Brooks System Manager - Detroit, MI Chapter 13 Standing Trustee - Tammy Terry Tom O’Hern Program Manager, ICF International, Baltimore, MD STACS - Standing Trustee Alliance for Computer Security STAFF SYMPOSIUM - IT TRACK SESSION 5 - DISASTER RECOVERY 1 4/14/2015

IT Track Outline Day 1 Session 1 (9:00 - 10:30) - HOW TO KEEP YOUR TRUSTEE HAPPY FROM AN IT PERSPECTIVE  Session 2 (10:45 - 12:15) - A RIVERWALK THROUGH YOUR NETWORK  Lunch (12:15-1:30) Session 3 (1:30 – 3:00) - DESKTOP & SERVER MANAGEMENT  Session 4 (3:15 – 4:45) - THE CLOUD – WHO REALLY UNDERSTANDS IT?  DAY 2 Session 5 (8:30 - 10:00) - DISASTER RECOVERY – YOUR WORST FEAR COMES TRUE  Session 6 (10:15 - 11:45) - BUERRITO BOWL - MIXED HOT TOPICS  List of Reference Material STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 2 4/14/2015

Sess ssion Focal Po Points ts  Disaster Recovery by Example  Backups and the Cloud  Server Virtualization STAFF SYMPOSIUM - IT TRACK SESSION 5 - DISASTER RECOVERY 3 4/14/2015

CryptoLocker: Office A  Staff member w/o administrator rights opens an infected email attachment containing what appears to be a valid Invoice  Other staff members report errors accessing encrypted files to the system manager  System manager see antivirus alerts on server and calls STACS  STACS helps identify infected computer which is removed from network (time <60 minutes)  Source of infection identified, no additional infections confirmed  Office reinstalls infected computer and restores prior night’s backup of encrypted files (< 48 hours) STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 4 4/14/2015

CryptoLocker: Office B  Staff Attorney with administrator rights opens an infected email attachment containing what appears to be a valid Invoice  Other staff members report errors accessing encrypted files to the system manager  System manager attempts to clean infection and restore encrypted files  Infected computer not identified and continues to encrypt shared files from server  Undiscovered secondary infection of SPAM bot malware occurs  Trustee has system manager contact STACS (1 week after infection discovered) STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 5 4/14/2015

CryptoLocker: Office B (continued)  STACS goes onsite (<12 hours), helps identify infected computer and removes from network  STACS helps correct missing/misconfigured antivirus, system patches, firewall, plan a backup/restore strategy  Office required to reinstall infected desktop and all servers, reinstall desktop security software on all systems (> 2 weeks)  Data lost and restoration of recoverable data (> 3 weeks) STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 6 4/14/2015

CryptoLocker: Office C  Administrator opens file attachment from staff email and leaves at the end of the day  Cryptolocker encrypts 10,000 files on shares from two servers over 16 hours  Trustee and staff discovery issues opening PDFs  Trustee has admin call STACS  < 15 minutes infected PC identified and removed from network.  Offsite backup service check to recovery files message reads “Server not in contact for 533 days.”  Administrator dismissed for gross negligence STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 7 4/14/2015

Contributing and Distinguishing Factors  User accounts configured with/without administrator rights  Systems with properly/poorly maintained antivirus & patches  Users quick/slow to notify system manager of unusual activity  System/office manager quick/slow to contact STACS for help  Backup and restoration process untested/tested & verified  Windows XP and 2003 Server still in use STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 8 4/14/2015

Disaster and Recovery Scenarios  Arbitrary deletion of file/folder discovered months later  SAN or NAS Failure / Server Failure ◦ Multi disk failure (RAID with Dual Parity) ◦ Controller failure leading to …  Corruption of Virtual machine(s) over 3-5 days ◦ (24-48 hours)  System Infection ◦ Requires a reformat of hard drives ◦ Requires a system rebuild or full system restore ◦ Requires full data restore – Data files and Databases STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 9 4/14/2015

What to backup? The Obvious  Bankruptcy data ◦ Case software database ◦ External case data files (PDFs) ◦ Case Software and configuration files  Expense Account data ◦ Software and license key ◦ Database encrypted export STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 10 4/14/2015

What to backup? The Not So Obvious  Electronic Disaster Recovery Kit ◦ Licenses  Microsoft Server and Seat licenses  Others purchased software license Keys ◦ Ex: Acrobat, antivirus,  Other Device Activation Keys ◦ Ex: Check printer, firewall  Backup solution license, serial numbers or keys ◦ Hard to get copies of software ◦ Encrypted Master Password list STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 11 4/14/2015

What to backup? System & Software Configuration ◦ Full System state (for bare metal restore) ◦ Server Software configurations  Web servers, ftp, email, exchange,  Exchange, IIS, SharePoint  Virtual Machines ◦ Device Configurations  Firewall configuration backup and firmware release  Wireless/Router configuration backup  Managed switch configuration – Especially with VLANs  NAS/SAN – especially RAID and logical volume setup  Hypervisor configuration STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 12 4/14/2015

What to backup? User Data  Email ◦ Hosted Mailbox – outsourced to 3rd party ◦ Hosted Temporarily – outsourced POPs/IMAPs ◦ Exchange – centralized mailbox folders ◦ Outlook – decentralized on user computer systems  Desktop Files ◦ User Profiles ◦ Documents and Settings STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 13 4/14/2015

What to backup? Organizational of data  File Shares ◦ Evaluate shares from all servers ◦ Backup agents need access to restricted folders  Accounting  Human Resources  System Admin (DR Kit)  Tax Returns ◦ AccessEnum  From Microsoft’s Sysinternals tool suite  Scans directories for file permissions STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 14 4/14/2015

Q&A  "Trustees, ask your system managers to answer these questions and show proof of the answers. Ex: backup status report, list of files and folders backed up. Review the list with you system manager to assure all critical data is backed up and recoverable if you lost everything last night. Ask what is not backed up, and review to make sure you can live without it. Otherwise get it on the backup list."  Any suggestions on how to prepare a backup list? How to prepare a NOT backed up list? I've been looking for some type of software that will prepare/save directory trees so that I can do this, but as yet have not found anything acceptable. Any suggestions here? Sure I can come up with some general details, - servers, drives, folders, etc, but it sounds like you want more than that - as do I. How about some specific suggestions on how to prepare these backup lists. Ideally, I would like a detail listing of everything and be able to flag what is or is not backed up, but I've been looking for the last few years and still haven't found a practical way to do this. STAFF SYMPOSIUM IT TRACK 4/14/2015 SESSION 5 - DISASTER RECOVERY 15

Types of Disaster Recovery Plans  No disaster plan at all  No disaster plan, but good backup procedures  A disaster plan, with no resources in place  A ‘cold site’ disaster recovery solution  A ‘split site’ or ‘warm site’ disaster recovery solution  A ‘hot site’ disaster recovery solution STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 17 4/14/2015

No disaster plan at all  No disaster plan; good backup procedures  The absolute minimum companies must do – even the smallest business – to prevent a disaster from wiping out business information is to back up the data on your computers daily and store the backups offsite at a secure archival company. Never store it at employee’s homes.  That way even if your hardware and software is ruined, you can still replace it and load it up with all your irreplaceable data. STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 18 4/14/2015

A disaster plan, with no resources in place  Once you have a good backup and archival procedure and your critical systems are fault tolerant, the next step is to put together procedures for remote disaster recovery.  This simply means you ask and answer the question, “What do we do if the computer center is utterly destroyed?”  Put your plan is written form and store a copy offsite STAFF SYMPOSIUM IT TRACK SESSION 5 - DISASTER RECOVERY 19 4/14/2015