iOS Multimedia CS 4720 Mobile Application Development CS 4720 iOS - PowerPoint PPT Presentation

iOS Multimedia CS 4720 – Mobile Application Development CS 4720

iOS Architecture CS 4720 2

Two Levels of Access • Some features live at a higher level in the main media player framework - MediaCoreServices • Fine-grained control is available in the AVFoundation framework • Most basic camera is in MediaCoreServices • Most audio/video in AVFoundation CS 4720 3

Aspects of Multimedia • Camera • Photo/Video Library • Audio Recording and Playback • Video Recording and Playback • Music Library • WebKit • Games / Animation CS 4720 4

Using the Camera • Check for camera availablility – Yes, most all iOS devices have a camera… • Typical use case: you want a user to take a picture from your app and then use the picture they took • Similar to Android – we are going to instantiate another part of the system to take care of this instead of writing our own CS 4720 5

Using the Camera • What circumstances would you want to use the camera? How would it enhance your app? CS 4720 6

Using the Camera import MobileCoreServices controller = UIImagePickerController() if let theController = controller{ theController.sourceType = .Camera theController.mediaTypes = [kUTTypeImage as String] theController.allowsEditing = true theController.delegate = self presentViewController(theController, animated: true, completion: nil) } CS 4720 7

Playing Back Video • Instead of MediaCoreServices, we now switch to the AVFoundationFramework • Lots of code for this… we will look at example on Github CS 4720 8

Available Codecs • Codec = Coder/Decoder • Some codecs are hardware enabled and some are software • Hardware codecs are MUCH more efficient – Power consumption – Speed – Processing power CS 4720 9

Codecs • Hardware Assisted – AAC (only recording format supported) – ALAC – MP3 • Software – All of the above – iLBC – PCM – Others CS 4720 10

Playing Audio • Try to use formats that have built-in decoding • But you can only play one at a time using the hardware! • Others are done via software CS 4720 11

Playing Audio var audioPlayer = AVAudioPlayer() let sound = NSURL(fileURLWithPath: NSBundle.mainBundle().pathForResource("tac-episode_114", ofType: "mp3")!) do{ print("Loading sound") let audioPlayer = try AVAudioPlayer(contentsOfURL:sound) print("Preparing to play") audioPlayer.delegate = self audioPlayer.prepareToPlay() print("Play") audioPlayer.play() }catch { print("Error getting the audio file") } CS 4720 12

Accessing the Music Library mediaPicker = MPMediaPickerController(mediaTypes: .AnyAudio) if let picker = mediaPicker{ print("Successfully instantiated a media picker") picker.delegate = self picker.allowsPickingMultipleItems = true picker.showsCloudItems = true picker.prompt = "Pick a song please..." view.addSubview(picker.view) presentViewController(picker, animated: true, completion: nil) } else { print("Could not instantiate a media picker") } CS 4720 13

Recording Audio let audioRecordingURL = self.audioRecordingPath() do { audioRecorder = try AVAudioRecorder(URL: audioRecordingURL, settings: audioRecordingSettings()) guard let recorder = audioRecorder else{ return } recorder.delegate = self /* Prepare the recorder and then start the recording */ if recorder.prepareToRecord() && recorder.record() CS 4720 14

WebKit • WebKit is an HTML browser engine • An engine takes HTML/CSS, processes it, and returns what should be displayed to the screen • Blink (Chrome) • Trident (IE) • Gecko (Firefox et al.) • WebKit (Safari) CS 4720 15

WebKit • The WebKit component is built into iOS in two ways: – Opening a link in Safari – Embedding the engine in your own application • When is the right time to do either option? • When is the right time to do neither? – (We’ll do more with this when we are done with iOS!) CS 4720 16

Games • Most all game control (if you are using default Apple libraries) is found in SpriteKit • A special Scene is loaded inside the ViewController • The ViewController still manages rotation, etc. but all control is passed to the game Scene CS 4720 17

GameScene override func touchesBegan(touches: Set<UITouch>, withEvent event: UIEvent?) { /* Called when a touch begins */ for touch in touches { let location = touch.locationInNode(self) let sprite = SKSpriteNode(imageNamed:"Spaceship") sprite.xScale = 0.5 sprite.yScale = 0.5 sprite.position = location let action = SKAction.rotateByAngle(CGFloat(M_PI), duration:1) sprite.runAction(SKAction.repeatActionForever(action)) self.addChild(sprite) } } CS 4720 18

Things to Remember • import AVFoundation or MediaCoreServices • Make sure your ViewController inherits from the correct delegate • Set the delegate correctly when media is being returned (i.e delegate = self) CS 4720 19

Using Media • Targetted and sparse usage • Have a real purpose for the usage – not just for show • Audio effects, in general, are a bad idea • Accessing a user’s media library is usually not the right idea (depends on feature/app) • Users want to feel that an app lives unto itself • Make sure to ask permission! CS 4720 20