1. Press Command-Shift-N and select Tab Bar Application in the iPhone OS | Application templates. Click Choose…
2. Provide a name for the project. I chose “Tab Test”
3. Click Save

The first thing you’ll notice is that the template has generated two classes for us. One called FirstViewController, which is used to control the first view (ya think?). The other is called Tab_TestAppDelegate which is our application delegate–the entry point and main controller class for our application.

Take a look at the stub code generated in the FirstViewController. It is commented out for the most part, but you’ll notice that there is a method in there that gives us what we want for allowing different orientations for our application. In particular, we are interested in -shouldAutorotateToInterfaceOrientation. Uncomment that code and have it simply return YES. This setting will enable the controller to support all orientations.

1
2
3
4
5

// Override to allow orientations other than the default portrait orientation.
- (BOOL)shouldAutorotateToInterfaceOrientation:(UIInterfaceOrientation)interfaceOrientation {
    // Return YES for supported orientations
    return YES;
}

This should cause our view to change orientation when the device is turned. Build and run the application using the simulator. Once the simulator has loaded the application, select Hardware | Rotate Left. You will notice that while the device did rotate, the view did not. Why is this?

All Or None

Remember that the default tab bar application template creates a tab bar component with two views. While we have explicitly told the FirstViewController object to allow all orientations, we have not done so with our second view. The way the tab bar controller works is that if any of the view controllers have a limitation on the orientation of the view, then all views are constrained by the same limitation.

What this means is that all view controllers must override the call to -shouldAutorotateToInterfaceOrientation to return the appropriate setting–YES to allow all. In our default tab bar app, we have not assigned the second view to a controller because we have not defined one, so where is it getting its controller from? Well, the answer is found in interface builder. If you double-click MainWindow.xib in the Resources group of the Xcode project, Interface Builder will load. Click on the tab for the first view in the main window, and then select the Identity tab in the Inspector, you’ll see what is going on.

FirstViewController

Notice in the first view Class field, it is set to FirstViewController. Take a close look now at our second view by clicking the second tab.

SecondView Controller

Notice that the class we’re using here is the base UIViewController class. This is our view’s controller. It only provides the base functionality–which is to say it doesn’t provide a whole lot. Furthermore it implements the default functionality. The default setting for the orientation of a view controller is portrait only. In other words, only derivative classes are going to be able to override -shouldAutorotateToInterfaceOrientation and set it to something other than the default.

If you want your application to be able to support more than the default portrait orientation, all of your tab views must have their own view controller that overrides -shouldAutorotateToInterfaceOrientation

Conclusion

The iPhone is a fun and interesting computing platform. It is very powerful, but it has a lot of nuances that make it fairly different from what you may be used to on OS X. The trick is to remember the principles you’ve already learned in Cocoa, MVC in particular, and you should be able to deal with the nuances and make them submit to your own will. Until next time.

Differentiating Keypath From Key

Each core animation layer contains a dictionary that enables you to add and remove animations based on a key that you define. I created a layer-backed view that receives mouse clicks and animates the position of the layer toward that mouse click.

1
2
3
4
5
6
7
8
9
10
11

- (void)mouseDown:(NSEvent*)event;
{
    CABasicAnimation *animation = [CABasicAnimation animationWithKeyPath:@"position"];
    [animation setDelegate:self];
    [animation setFromValue:[NSValue valueWithPoint:previousValue]];
    previousValue = NSMakePoint([event locationInWindow].x, [event locationInWindow].y);
    [animation setToValue:[NSValue valueWithPoint:previousValue]];
    [animation setDuration:2.0];
 
    [objectLayer addAnimation:animation forKey:@"follow"];
}

Notice that when I create the animation, I am creating it with a keypath–in this case position. The keypath is different from the key by which we store the animation in the animations dictionary contained by the layer. In the sample code I add the animation to the dictionary using a key called “follow”. This could be any string that is meaningful to keep track of the animation in question. Consider it a tag.

I only make this distinction because I’ve seen some confusion about the difference between the two. To summarize, the keypath specifies which layer property you want to animate while the key is the key by which you will store, recall, and delete an animation from the animation dictionary contained by the layer.

Adding The Animation, Again

In the code above, you’ll also notice that we are adding the animation to the layer for the same key each time we receive a -mouseDown. If we are setting a new object in the dictionary each time, then how are we able to keep track of whether or not the animation finished?

By setting a delegate for the animation, we can be notified when animation has been removed from the animations dictionary. We implement the -animationDidStop:finished delegate.

1
2
3
4

- (void)animationDidStop:(CAAnimation *)theAnimation finished:(BOOL)flag
{
    NSLog(@"Animation interrupted: %@", (!flag)?@"Yes" : @"No");
}

There are several things going on here. When we call [objectLayer addAnimation:animation forKey:@"follow"]; in our -mouseDown code, the animation that was there previously, if there was one, is replaced by the new one. If the user clicked on the screen before the animation completed, the animation is removed and the delegate -animationDidStop gets called. If instead of clicking the mouse again before the current animation is finished, you just let it finish, the animation is also removed from the dictionary because this is the default behavior for an animation. If you don’t specify [animation setRemovedOnCompletion:NO] explicitly, the animation will be removed.

Changing Direction While In Flight

When the view receives a click the basic animation is re-created and added to the layer for the key “follow” again. The fromValue field needs set again, but if you click another spot on the view before the animation finishes, what value do you set it to? If you set it to the previous value, it will simply animate from the previous location which won’t look right. You want it to change direction rather than starting from the previous start location.

If instead of using a basic animation in our example code we had simply opted to use the animator proxy, calling -setPosition on the layer would do this for us. The problem is that we want to be able to demonstrate the -animationDidStop delegate which we can’t do with the animator proxy. So how do we provide this functionality of changing direction in flight?

Look no further than the presentationLayer which is accessible through our layer object. We can obtain the last location of our “position” parameter by first getting the presentationLayer and getting the position from it instead. We’ve modified our -mouseDown event to reflect this change.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

- (void)mouseDown:(NSEvent*)event;
{
  CABasicAnimation *animation = 
          [CABasicAnimation animationWithKeyPath:@"position"];
  [animation setDelegate:self];
 
  // Get the presentationLayer
  CALayer *p = (CALayer*)[objectLayer presentationLayer];
  CGPoint position = [p position];
  NSValue *prevVal = [NSValue valueWithPoint:
                      NSPointFromCGPoint(position)];
 
  [animation setFromValue:prevVal];
 
  [animation setToValue:[NSValue valueWithPoint:[event locationInWindow]]];
  [animation setDuration:2.0];
  [animation setRemovedOnCompletion:NO];
 
  [objectLayer addAnimation:animation forKey:@"follow"];
}

Now, when we click in our view, the animation changes directions from its current location in the animation. The presentationLayer provides you with current values of whatever field you are currently animating. If a value is not being animated, you will get the same value as what is in the main layer.

Conclusion

It’s very handy to have a way to know when an animation is completed. You simply set your app delegate to be your animation’s delegate and when the animation completes, -animationDidStop:finished gets called conveying through the finished flag whether the animation was interrupted or not. To get our in flight values when the animation hasn’t completed, we simply look at the presentationLayer. These features of Core Animation are very powerful. Given the choice I would like to see the presentationLayer values become KVO compliant as this would let you monitor changes in real time, but being able to read the current value at any given time is a great capability in its own right. It’s very cool technology. Until next time.

Follow Me Demo Project

While new Cocoa programmers will find it a great help to getting started quickly with Core Data, the book also covers some really interesting and advanced topics such as data versioning and migration, Spotlight/Quick Look integration, Sync Services, and multi-threading. You can really see Marcus’ command of the subject shine in these chapters which are already available in the beta.

Give Marcus some feedback on the book as it progresses. It’s going to be a great reference for any Cocoa Developer looking to harness the power of Core Data.

$21.00 for the beta PDF
$41.35 for the beta PDF plus hard copy when it’s released in March.

Mad props to Marcus. Congratulations!

The Slider Value

We connect a slider to an IBAction in xcode that will get called continuously as we have checked the checkbox called ‘Continuous’ in Interface Builder. We can read the slider value and apply it to our animation that we use to rotate the layer. In interface builder, we have specified a minimum value of -180, a maximum value of 180 and a current value (our starting value) of 0 on the slide control. When the slider is moved, the rotation updates as does the value of our text layer.

While it is coincidental that the demo application sort of looks like a pumpkin, it seems appropriate considering the time of year.

Layer Rotate

Rotation Animation

Rotating a layer is very simple. You just need to create an animation with “transform.rotation.z” as your keypath and then you can set from and to values that the animation will rotate between. We simply keep track of the last value we rotated (previousValue in the code) to and use this as our from value in the next change to the slider.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

- (CAAnimation*)rotateAnimation;
{
  CABasicAnimation * animation;
  animation = [CABasicAnimation 
               animationWithKeyPath:@"transform.rotation.z"];
 
  [animation setFromValue:DegreesToNumber(previousValue)];
  [animation setToValue:DegreesToNumber([slider floatValue])];
 
  [animation setRemovedOnCompletion:NO];
  [animation setFillMode:kCAFillModeForwards];
 
  previousValue = [slider floatValue];
 
  return animation;
}

When our -sliderChange function gets called when the user moves the slider, we add the animation to the layer again using the key “rotate”. This is done first in our -awakeFromNib call and then in the -sliderChange call.

1
2
3
4
5
6

- (IBAction)sliderChange:(id)sender;
{
  [degreesTextLayer setString:[NSString stringWithFormat:@"%f", [slider floatValue]]];
  [needleLayer addAnimation:[self rotateAnimation] forKey:@"rotate"];
  [needleLayer setNeedsDisplay];
}

You will notice that we call -setNeedsDisplay on the layer. This is necessary as we have implemented the delegate method -drawLayer:inContext. We use this method to draw the vertical and horizontal black lines that intersect in the center of the circle layer.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

- (void)drawLayer:(CALayer *)layer inContext:(CGContextRef)context;
{
  if( layer == needleLayer )
  {
      CGMutablePathRef path = CGPathCreateMutable();
 
      CGPathMoveToPoint(path,NULL, [view bounds].size.width/2.0, 0.0);
      CGPathAddLineToPoint(path, NULL, [view bounds].size.width/2.0, [view bounds].size.height);
 
      CGPathMoveToPoint(path,NULL, 0.0, [view bounds].size.height/2.0);
      CGPathAddLineToPoint(path, NULL, [view bounds].size.width, [view bounds].size.height/2.0);
 
      CGColorRef black =
              CGColorCreateGenericRGB(0.0, 0.0, 0.0, 1.0);
      CGContextSetStrokeColorWithColor(context, black);
      CFRelease(black);
 
      CGContextBeginPath(context);
      CGContextAddPath(context, path);
 
      CGContextSetLineWidth(context, 3.0);
 
      CGContextStrokePath(context);
 
      CFRelease(path);
 
  }  
}

Retaining The Rotation Position

There are two parameters of an animation that must be set otherwise the layer transform will just revert back to the original rotation. These two lines of code in our animation creation code ensure that what you see remains sticky when the value is changed.

[animation setRemovedOnCompletion:NO];
[animation setFillMode:kCAFillModeForwards];

Now, when the slider moves, the vertical and horizontal lines will stay in the correct rotation position. If you comment those two lines of code out in the demo project, you will see that while the rotation layer animates, it jumps right back to its original rotation as soon as the animation completes.

Conclusion

You will likely find many uses for this type of control. You could of course control the rotation based upon some other value or control other than a slider, but this code demonstrates how simple it is to rotate a layer based upon user input. Until next time.

Layer Rotate Demo Project

If you have not heard, O’Reilly is hosting a conference on November 18, 2008 to discuss all things iPhone. I have been invited to attend the conference as a Panelist.

Please come and join the conference, if nothing else, to heckle me :)

iPhone Live

The list of speakers (as opposed to panelists), is quite impressive and definitely worth the trip.

Speakers

If you are a visual learner, it may be helpful to you to instead watch a video presentation of this tutorial. I’ve posted it on the site, but you’ll have to click the link to see my Cocoa Touch Video Tutorial.

Understanding Cocoa programming is much simpler if you learn MVC, Model, View, Controller. You can probably step through code examples and figure some things out without learning MVC, but I wouldn’t recommend it. Go Google it and read up on it.

I will say as an introduction to MVC for those who are not familiar that it should probably be called (Model <--> Controller <--> View) or (View <--> Controller <--> Model) as the controller always sits between the other two. Your controller is either telling your model to update its data or it is telling the view to update its display. That’s the crux of the whole paradigm. The details run much deeper, but that’s how I will nutshell it for you.

Create Your Application

Let’s get started. Create a Cocoa Application using the following steps:

1. Select File > New Project…, under the iPhone OS templates choose Window-Based Application in the ensuing dialog. Click Choose…



2. Enter ‘Basic iPhone App’ as the project name. Click Save

You should see a project workspace like the following:

The next thing you should do is create a class to act as your controller or delegate.

Delegate == Controller

The words delegate and controller can be used synonymously. You’ll see later that we delegate the work of the different controls we create in Interface Builder to a delegate or controller class. In the iPhone template projects, this application delegate is created for you. Our app delegate has been named Basic_iPhone_AppAppDelegate.

In our app delegate class we need to add what Cocoa developers refer to as outlets and actions. I could spend an entire post explaining these two things in depth, but for the sake of brevity and walking you through the steps to build your first application, the definition will have to suffice.

Outlets represent controls in your user interface that can have some action performed upon them. Action are functions in your code that are connected to controls in your user interface such as a button or a drop down list. When connected to a button for instance, the action code will be run when the user clicks the button.

In xcode, open your app delegate header file Basic_iPhone_AppAppDelegate.h. Add an outlet for the text field and the label below the window outlet as in the following snippet:

1
2
3
4
5

@interface Basic_iPhone_AppAppDelegate : NSObject <UIApplicationDelegate> {
    IBOutlet UIWindow *window;
    IBOutlet UITextField *textField;
    IBOutlet UILabel *label;
}

You will also want to add an action that will be performed when our button is clicked. Add that below the property for our window:

1
2
3
4
5
6
7
8
9

@interface Basic_iPhone_AppAppDelegate : NSObject <UIApplicationDelegate> {
    IBOutlet UIWindow *window;
    IBOutlet UITextField *textField;
    IBOutlet UILabel *label;
}
 
@property (nonatomic, retain) UIWindow *window;
 
- (IBAction)click:(id)sender;

1
2
3
4
5
6
7
8
9
10

- (void)applicationDidFinishLaunching:(UIApplication *)application {	
 
    // Override point for customization after app launch	
    [window makeKeyAndVisible];
}
 
- (IBAction)click:(id)sender;
{
 
}

Interface Builder And Controller/Delegate Implementation

Now that you’ve specified the outlets–a UITextField and a UILabel and an action called click:, you will see these items available for connecting to the UI in Interface builder. Let’s open Interface Builder and make the connections we need using the following steps:

1. In your xcode workspace, expand the folder in the tree view called Resources and double click the file called ‘MainMenu.xib’.
   

   Note:: A .xib is a .nib that uses XML for the internal data structure.

   This will open the xib file in Interface Builder

2. Once Interface Builder loads, you will notice that there is an object representation of our app delegate in the MainWindow.xib window. This is the object you will use to create connections for your actions and outlets.

   

Design The User Interface

Now you simply need to add the controls to the main window in Interface Builder and then we can connect the action and outlet accordingly. To finish the interface, complete the following steps:

1. Drag a TextField, a Label, and a Button to the main window so that the user interface looks like the screenshot below:

   

2. Control-Click and drag from the Button to your app delegate object in the ‘MainWindow.xib’ window.

   

   A pop-up will display. Select click:

3. Control-Click the app delegate object and drag it to the text field in the main window.

   

   A pop-up will display. Select textField

4. Control-Click the app delegate object and drag it to the label in the main window.

   

   A pop-up will display. Select label

That’s it for Interface Builder. You can quit interface builder and return to xcode. We have one more piece of code to add and then our application will be finished.

Finishing Up

When the button is clicked, it will simply grab the text from the text field and place it into the label. That’s all the application does. Here’s the code you need. Just make your implementation of the click: action in the Basic_iPhone_AppAppDelegate.m file look like this:

1
2
3
4

- (IBAction)click:(id)sender;
{
    [label setText:[textField text]];
}

Notice we are grabbing the text from the text field, and setting the text in our label with it. Now all you need to do is click “Build and Go”. When the application runs the iPhone Simulator will be started. You will see the application load. Type some text into the text field and click the Change button. You will see the label update with the text from the text field.

Next Steps

This tutorial has just scratched the surface. It’s is intended to get you going quickly. For a more in-depth path to gaining a deeper understanding of iPhone development, take a look at Apple’s, Your First iPhone Application. You will need to log in with your ADC account to see the article, but it is definitely worth while.

Conclusion

The limit with iPhone development is really just your imagination. It is an extremely fun platform to develop on and the resulting applications are very rewarding. Have fun with it and learn as much as you can. Just do me a favor and don’t clog the App Store with any more flashlight apps or tip calculators… Seriously ;-) . Until next time.

Basic iPhone Application Demo Project

1
2
3
4
5
6
7

[[window contentView] setWantsLayer:YES];
[qcView setWantsLayer:YES];
 
CATextLayer *text = [CATextLayer layer];
[text setString:@"hello World"];
[text setFrame:NSRectToCGRect([qcView frame])];
[[qcView layer] addSublayer:text];

This code would allow the text layer to show up, however, the Quartz Composition was gone and all I got was a gray window with the “hello World” text on it.

Two Heads Better Than One

This is what I love about NSCoder Night. You get some Cocoa geeks together and the collective wisdom starts this synergy where we can bounce things off each other and a solution emerges.

Chad, in the mean time, had been searching the web for some examples and at about the time I started to wonder if there was a Core Animation specific layer for Quartz Compositions, he said, “QCCompositionLayer”. Marcus and I were both at that point thinking, “ah, I should have thought of that since there are Core Animation layers to do everything else media related.”

I then went to town slinging some code and here is what I came up with:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

- (void)awakeFromNib;
{
  [[window contentView] setWantsLayer:YES];
 
  QCCompositionLayer *layer = [[QCCompositionLayer alloc] 
            initWithFile:@"/Developer/Examples/Quartz Composer/Plugins/OpticalFlow/Teapotii.qtz"];
 
  [layer setFrame:NSRectToCGRect([[window contentView] frame])];
 
  CATextLayer *text = [CATextLayer layer];
  [text setString:@"hello World"];
  [text setFrame:NSRectToCGRect([[window contentView] frame])];
  [ayer addSublayer:text];
 
  [[[window contentView] layer] addSublayer:layer];    
}

I got rid of the QCView altogether and just added the Quartz Composer layer to the sublayers of the window’s content view. The Quartz Composition in a QCCompositionLayer begins to play automatically, so really this code is all there is to it. And notice that the text overlay works as well. You could really add anything you wanted in a Core Animation layer at this point.

Your Own Composition

If you’re anything like me this will whet your appetite for playing with some very cool technology. I decided to create my own Quartz Composition and include that in the demo project. The demo composition takes the vide input of your iSight camera and applies the image to each side of a cube. It then rotates the cube. Creating the composition was pretty easy with a cursory reading of Apple’s Quartz Composer User Guide. It’s quite helpful.

You can take a look at the Quartz Composition in Quartz Composer. The demo application has two .qtz files in it. You can just double click either of those from the Finder to see what I did. The one I’m using in the demo app is the one called “VideoCub.qtz”. You can switch it in the code to see the other one as well, which is a quad of four iSight inputs.

VideoCube QC Editor

Quartz Composer Layer

You can see I added a CATextLayer to the QCCompositionLayer which displays the words, “hello World”. I also applied a Gaussian blur filter to the whole thing so you can see how easy it is to add filters as well. Here is *all* of the code needed to make this work.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

- (void)awakeFromNib;
{
    [[window contentView] setWantsLayer:YES];
 
    NSString *path = [[NSBundle mainBundle] 
                      pathForResource:@"VideoCube" 
                      ofType:@"qtz"];
 
    QCCompositionLayer *layer = 
        [[QCCompositionLayer alloc] initWithFile:path];
 
    [layer setFrame:NSRectToCGRect([[window contentView] frame])];
 
    CATextLayer *text = [CATextLayer layer];
    [text setString:@"hello World"];
    [text setFrame:NSRectToCGRect([[window contentView] frame])];
    [layer addSublayer:text];
 
    CIFilter *blurFilter = 
    [CIFilter filterWithName:@"CIGaussianBlur"];
 
    [blurFilter setDefaults];
    [blurFilter setValue:[NSNumber numberWithFloat:2.0] 
                  forKey:@"inputRadius"];
    [blurFilter setName:@"blur"];
 
    [layer setFilters:[NSArray arrayWithObject:blurFilter]];
 
 
    [[[window contentView] layer] addSublayer:layer];
}

What’s even more exciting to me is that if you know a bit of OpenGL, you can do all kinds of other manipulations of the image since QCCompositionLayer inherits from CAOpenGLLayer. Take a close look at -drawInCGLContext in the CAOpenGLLayer API docs. There is a ton of great potential there.

Conclusion

The conclusion is that Quartz Compositions are very cool and they are now even cooler since you can work with them in Core Animation. Take a look a the Quartz Composer guide and start building your own compositions that you can easily incorporate into your Core Animation based applications. Until next time.

Quartz Composition Layer Demo Project

Why are these topics linked?

Generally, there are a few ways to add plug-ins to a Cocoa application.

Scriptable plug-ins

Scriptable plug-ins are probably the easiest to add to an application. They exist in a known location where the application can load them and run them in the appropriate virtual machine. This makes them fairly easy to construct even though the code to run them may be rather complicated.

Publishing a Formal Protocol Plug-ins Interface

This one is a bit more complicated. The developer (you and me) publish a formal protocol that the plug-ins must adhere to. This design works rather well but does not allow us, the application developer, to impart any functionality to the plug-ins easily.

Publishing an Informal Protocol Plug-ins Interface

This is very similar to the formal plug-ins above but a little more tricky. Since the protocol is informal, we have to test for each method that we want to use in the plug-ins before we can use it. Again, it does not make it very easy for us to impart functionality to the plug-ins.

Publishing a Base Class Plug-ins Interface

This is the hardest option I have found so far. Not only do we have to give our plug-ins developers a header file but something to actually compile against. However, with a base class we can easily impart functionality to our plug-ins without the plug-ins developer having to jump through too many hoops.

Creating a base class solution is where frameworks come in. It makes it fairly trivial to hand off a base class to plug-ins developers when we give them a framework. Therefore, we are going to add one class to the plug-ins we wrote previously with the following header:

#import <Cocoa/Cocoa.h>
 
@interface AbstractPlugin : NSObject 
{
 
}
 
- (NSString*)name;
 
- (IBAction)fire:(id)sender;
 
@end

And since I like to make it stupidly easy for people, we are going to use the following implementation of the base “abstract” class:

#import "AbstractPlugin.h"
 
#define kErrFormat @"%s not implemented in subclass %@"
#define kExceptName @"CIMGF"
 
@implementation AbstractPlugin
 
- (NSString*)name;
{
  NSString *reason = [NSString stringWithFormat:kErrFormat, 
    _cmd, [self class]];
  @throw [NSException exceptionWithName:kExceptName 
                                 reason:reason 
                               userInfo:nil];
}
 
- (IBAction)fire:(id)sender;
{
  NSString * reason = [NSString stringWithFormat:kErrFormat, 
    _cmd, [self class]];
  @throw [NSException exceptionWithName:kExceptName 
                                 reason:reason 
                               userInfo:nil];
}
 
@end

For every method that the subclass must override the abstract class will throw an exception if it is not overwritten with a polite message explaining what method in which class is missing.

Building the plug-ins

A plug-ins is effectively just a bundle, like a framework or an application. Inside of the bundle is a Contents directory, MacOS directory (where the binary goes) and a Resources directory.

Xcode has a template for building plug-ins that we can use to get started. For this first plug-ins, we are going to just add it to our existing project as a new target.

This new target is very similar to the Framework target that we built previously. We want to drag it up into the Application’s target so that it is a dependency; this can also be done from the Get Info window. We will also want to add the Framework product as a linked framework in the plug-ins. The end result looks like this.

Once we have the target set up, it is time to add some code. Plug-ins needs to have either a main class defined or a main nib defined. It is this definition that helps determine how to launch the bundle. In our case, we will want to define a main class which will be explained below. To define a main class (or nib), we need to look at the properties tab of the target.

The next step is to create the main class for our plug-ins. Since we are using a base class design for our plug-ins, our main class must extend the AbstractPlugin class and it needs to import the framework directly in the header. For our example, our plug-ins is going to display a new window on the screen, therefore the header must also include a NSWindow attribute and since we want to programmatically close the window, a -closeWindow: IBAction.

#import <Cocoa/Cocoa.h>
#import <Example/Example.h>
 
@interface PluginMainClass : AbstractPlugin 
{
  IBOutlet NSWindow *mainWindow;
}
 
- (IBAction)closeWindow:(id)sender;
 
@end

Note that we are importing the framework which contains the AbstractPlugin super class we built earlier.

Once we have the main class defined it is time to build the user interface to go along with this plug-ins. It is not a requirement that plug-ins have a user interface, they can be pure code instead. However for the sake of completeness, we are going to add one for this example.

The structure of the plug-ins nib is simple. The file owner is the PluginMainClass we wrote previously and there is one window which we bind to the file owner.

The window is also equally simple, just one button that binds back to the file owner’s -closeWindow: method.

The last step is to add a copy phase to our application. Like the framework copy phase we built last time, this is a “Copy Files” build phase and we need to configure it to copy files to the “PlugIns” directory. This is another top level directory next to Resources and Frameworks. Once the build phase is added, drag the plug-ins product into that build phase.

With the window and the main class built we can compile the plug-ins and make sure everything is ready for inclusion into our application. Once the app is built, we should see a copy of the plug-ins inside of the application bundle. Now it is time to dynamically load the bundle.

The flow of loading bundles dynamically is as follows:

1. Find the paths of the bundles.
2. Instantiate NSBundle objects for the paths.
3. Execute the -load method on the bundle.
4. Retrieve the principal class from the bundle.
5. Instantiate an object from the principal class.

And the code for this is as follows:

- (NSArray*)loadPlugins
{
  NSBundle *main = [NSBundle mainBundle];
  NSArray *all = [main pathsForResourcesOfType:@"bundle" 
                                   inDirectory:@"../PlugIns"]];
 
  NSMutableArray *availablePlugins = [NSMutableArray array];
 
  id plugin = nil;
  NSBundle *pluginBundle = nil;
 
  for (NSString *path in all) {
    pluginBundle = [NSBundle bundleWithPath:path];
    [pluginBundle load];
 
    Class prinClass = [pluginBundle principalClass];
    if (![prinClass isSubclassOfClass:[AbstractPlugin class]]) {
      continue;
    }
 
    plugin = [[prinClass alloc] init];
    [availablePlugins addObject:plugin];
    [plugin release], plugin = nil;
    pluginBundle = nil;
  }
  return availablePlugins;
}

In this new method that we have added to the AppDelegate we first get a reference to the main bundle and we then request a path for every file in the PlugIns directory that has an extension of bundle. With that array in hand we then start to loop over the array and construct a new NSBundle for each path. Once the bundle is constructed we ask for it to load. After the load, we grab its principal class, check to make sure we can use it, and construct an object from it.

Since we want to have a list of the loaded plug-ins, we throw this new object into an array that will be returned back from the method. Once this method is complete we now have every plug-ins dynamically loaded and ready to use. So lets use them!

We really want to call this method before the application has finished loading, therefore we need to add an -init method to our AppDelegate and load the plug-ins there. This will guarantee that they are loaded before the UI with some obvious benefits.

- (id)init
{
  if (![super init]) return nil;
 
  [self setPlugins:[self loadPlugins]];
 
  return self;
}

Here are we calling our new -loadPlugins method and setting the returned array into a ivar. This ivar will then be used by our UI once it loads to display a list of the plug-ins.

The main window has a pop up button with a list of the plug-ins using their inherited -name method for display. When the run button is clicked, our application will find out which plug-ins were selected via the NSArrayController and call the inherited -run: method on the plug-ins with predictable results.

Conclusion

For clarity we did leave a few things out of this project. For example, there is no code for unloading the plug-ins nor do we search the Application Support directory for user added plug-ins. However these are easy to add based on the code that was included.

I should also note that we used the default extension for these plug-ins. I would recommend changing this extension to something unique to your application and add that extension (or UTI) to your main application as an accepted file type. See previous articles here on CIMGF to see how to do each of those.

Example Project

Getting two technologies to work together can be a challenge especially when the bridges between the two are not necessarily clearly laid out in documentation. As I pointed out Apple provides some excellent sample code to help you along, but there is no hand-holding approach to any of it. I actually appreciate that having come from the Windows world as it seems that there all you get sometimes is hand-holding where Microsoft doesn’t trust you, the developer to figure things out and really own what you’re doing. But I digress (wouldn’t be a CIMGF post if I didn’t dig on MS a bit).

Like peanut butter and chocolate, what you get when you put together QuickTime and OpenGL is something greater than either of them left on their own (ok, this is subjective. Not everyone likes peanut butter and chocolate together, but again, I digress).

If you read the Core Video Programming Guide from Apple, you see they provide the reasons for using Core Video:

CoreVideo is necessary only if you want to manipulate individual video frames. For example, the following types of video processing would require CoreVideo:

• Color correction or other filtering, such as provided by Core Image filters
• Physical transforms of the video images (such as warping, or mapping on to a surface)
• Adding video to an OpenGL scene
• Adding additional information to frames, such as a visible timecode
• Compositing multiple video streams

If all you need to do is display a movie, you should simply use either a QTMovieView or, if you want to stick with the Core Animation route, use a QTMovieLayer. They both function similarly, however, the view provides a lot of features that you won’t have to implement in the UI yourself such as a scrubber or play/pause buttons. Plus the view is very fast and efficient. I’m in the process of exploring performance differences between the two, but I will save my comments about that for another post.

For our example code we are most interested in the third point above–adding video to an OpenGL scene. It seems that new QuickTime developers often want to know how to manipulate movie images before displaying them. Often this leads them to pursue adding sub-views to the movie view which can become a big mess. Because we are using OpenGL, doing other drawing on the scene is very fast. I won’t kid you. OpenGL is a pain. I don’t know anybody who loves it, but everybody respects it because of its raw speed.

Point number five above–compositing multiple video streams–is also interesting. While I won’t be covering it in this post, I will say that it makes a world of difference performance wise if you composite the movies into a OpenGL scene. If you’ve ever tried to run multiple videos simultaneously in two different views or layers, it can get pretty herky jerky. You can see why it is necessary to use OpenGL instead.

The OpenGL QuickTime Two Step

Ok, it actually will take more than two steps, however, when you are working with Core Animation layers things get a whole lot easier than they are for rendering a movie in an NSOpenGLView. Here is what you get for free, as the kids say.

• You don’t have to set up the OpenGL context. It is already available for you to send your OpenGL calls to.
• The viewport for display is already configured
• You don’t need to set up a display link callback

What took over 400 lines of code when rendering a QuickTime movie with no filters to an NSOpengGLView now only takes around 150 lines. Any time you can reduce code to something simpler, it makes life easier. It also makes it much easier to grok, in my opinion.

There really are two primary steps you take when using a CAOpenGLLayer. First you check to see if you should draw. Then, depending upon the answer, drawInCGLContext gets called or doesn’t. Really thats it. Determining whether or not you should draw depends upon what you are trying to do. In our case, we only want to draw if all of the following are true:

• The movie is actually playing back
• The visual context for the movie has been initialized
• The visual context has a new image ready to be rendered

If all of these are true, then our call to canDrawInCGLContext returns YES. Here is the code I use to check these contraints in canDrawInCGLContext:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

- (BOOL)canDrawInCGLContext:(CGLContextObj)glContext 
                pixelFormat:(CGLPixelFormatObj)pixelFormat 
               forLayerTime:(CFTimeInterval)timeInterval 
                displayTime:(const CVTimeStamp *)timeStamp
{ 
    // There is no point in trying to draw anything if our
    // movie is not playing.
    if( [movie rate] <= 0.0 )
        return NO;
 
    if( !qtVisualContext )
    {
        // If our visual context for our QTMovie has not been set up
        // we initialize it now
        [self setupVisualContext:glContext withPixelFormat:pixelFormat];
    }
 
    // Check to see if a new frame (image) is ready to be draw at
    // the time specified.
    if(QTVisualContextIsNewImageAvailable(qtVisualContext,timeStamp))
    {
        // Release the previous frame
        CVOpenGLTextureRelease(currentFrame);
 
        // Copy the current frame into our image buffer
        QTVisualContextCopyImageForTime(qtVisualContext,
                                        NULL,
                                        timeStamp,
                                        &currentFrame);
 
        // Returns the texture coordinates for the part of the image that should be displayed
        CVOpenGLTextureGetCleanTexCoords(
                            currentFrame, 
                            lowerLeft, 
                            lowerRight, 
                            upperRight, upperLeft);
        return YES;
    }
 
    return NO;
}

The call to setup the visual context is where we are associating the QuickTime movie itself with a QTVisualContextRef object which is what OpenGL needs to draw the current frame. We will then use this object to load image data into a CVImageBufferRef which can be used for rendering with OpenGL. Here is the code to set up the visual context.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

- (void)setupVisualContext:(CGLContextObj)glContext 
           withPixelFormat:(CGLPixelFormatObj)pixelFormat;
{
    OSStatus			    error;
 
    NSDictionary	    *attributes = nil;
    attributes = [NSDictionary dictionaryWithObjectsAndKeys:
                  [NSDictionary dictionaryWithObjectsAndKeys:
                   [NSNumber numberWithFloat:[self frame].size.width],
                   kQTVisualContextTargetDimensions_WidthKey,
                   [NSNumber numberWithFloat:[self frame].size.height],
                   kQTVisualContextTargetDimensions_HeightKey, nil], 
                  kQTVisualContextTargetDimensionsKey, 
                  [NSDictionary dictionaryWithObjectsAndKeys:
                   [NSNumber numberWithFloat:[self frame].size.width], 
                   kCVPixelBufferWidthKey, 
                   [NSNumber numberWithFloat:[self frame].size.height], 
                   kCVPixelBufferHeightKey, nil], 
                  kQTVisualContextPixelBufferAttributesKey,
                  nil];
 
    // Create our quicktimee visual context
    error = QTOpenGLTextureContextCreate(NULL,
                                         glContext,
                                         pixelFormat,
                                         (CFDictionaryRef)attributes,
                                         &qtVisualContext);
 
    // Associate it with our movie.
    SetMovieVisualContext([movie quickTimeMovie],qtVisualContext);
}

Next we check to see if there is an image ready using:

1

if(QTVisualContextIsNewImageAvailable(qtVisualContext,timeStamp))

And then we copy the image to our CVImageBufferRef with:

1
2
3
4
5

// Copy the current frame into our image buffer
QTVisualContextCopyImageForTime(qtVisualContext,
                                NULL,
                                timeStamp,
                                &currentFrame);

Now it’s all a matter of rendering the frame for the current time stamp.

But Wait! What TimeStamp?

If you asked this question, then you are a very astute reader. In order to obtain the next image, we simply passed the CVTimeStamp parameter, timeStamp to our call to QTVisualContextCopyImageForTime. But how do we even have a timestamp? Isn’t that something we need to get from a display link? If you’re asking what is a display link at this point, take a look at the Core Video Programming Guide which states:

To simplify synchronization of video with a display’s refresh rate, Core Video provides a special timer called a display link. The display link runs as a separate high priority thread, which is not affected by interactions within your application process.

In the past, synchronizing your video frames with the display’s refresh rate was often a problem, especially if you also had audio. You could only make simple guesses for when to output a frame (by using a timer, for example), which didn’t take into account possible latency from user interactions, CPU loading, window compositing and so on. The Core Video display link can make intelligent estimates for when a frame needs to be output, based on display type and latencies.

I will provide a more complete answer to the question in the future as I am still studying it myself, however, I will mention that a display link callback is unnecessary in this context as the CAOpenGLLayer is providing this for us. The timestamp field is all we need in order to get the current frame assuming that the movie is playing back.

Drawing The Frame

There is a special group of people who really get OpenGL. I salute all of you to whom this applies. You are amazing. I, however, only write as much of it as necessary and you’ll see that most of the code I have here is simply a copy and paste from sample code I got from Apple. I am starting to understand it more and more, however, it makes my brain hurt. Here is my drawing code for when a frame is ready to be rendered.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

- (void)drawInCGLContext:(CGLContextObj)glContext 
             pixelFormat:(CGLPixelFormatObj)pixelFormat 
            forLayerTime:(CFTimeInterval)interval 
             displayTime:(const CVTimeStamp *)timeStamp
{
    NSRect    bounds = NSRectFromCGRect([self bounds]);
 
    GLfloat 	minX, minY, maxX, maxY;        
 
    minX = NSMinX(bounds);
    minY = NSMinY(bounds);
    maxX = NSMaxX(bounds);
    maxY = NSMaxY(bounds);
 
    glMatrixMode(GL_MODELVIEW);
    glLoadIdentity();
    glMatrixMode(GL_PROJECTION);
    glLoadIdentity();
    glOrtho( minX, maxX, minY, maxY, -1.0, 1.0);
 
    glClearColor(0.0, 0.0, 0.0, 0.0);	     
    glClear(GL_COLOR_BUFFER_BIT);
 
    CGRect imageRect = [self frame];
    // Enable target for the current frame
    glEnable(CVOpenGLTextureGetTarget(currentFrame));
    // Bind to the current frame
    // This tells OpenGL which texture we are wanting 
    // to draw so that when we make our glTexCord and 
    // glVertex calls, our current frame gets drawn
    // to the context.
    glBindTexture(CVOpenGLTextureGetTarget(currentFrame), 
                  CVOpenGLTextureGetName(currentFrame));
    glMatrixMode(GL_TEXTURE);
    glLoadIdentity();
    glColor4f(1.0, 1.0, 1.0, 1.0);
    glBegin(GL_QUADS);
 
    // Draw the quads
    glTexCoord2f(upperLeft[0], upperLeft[1]);
    glVertex2f  (imageRect.origin.x, 
                 imageRect.origin.y + imageRect.size.height);
    glTexCoord2f(upperRight[0], upperRight[1]);
    glVertex2f  (imageRect.origin.x + imageRect.size.width, 
                 imageRect.origin.y + imageRect.size.height);
    glTexCoord2f(lowerRight[0], lowerRight[1]);
    glVertex2f  (imageRect.origin.x + imageRect.size.width, 
                 imageRect.origin.y);
    glTexCoord2f(lowerLeft[0], lowerLeft[1]);
    glVertex2f  (imageRect.origin.x, imageRect.origin.y);
 
    glEnd();
 
    // This CAOpenGLLayer is responsible to flush
    // the OpenGL context so we call super
    [super drawInCGLContext:glContext 
                pixelFormat:pixelFormat 
               forLayerTime:interval 
                displayTime:timeStamp];
 
    // Task the context
    QTVisualContextTask(qtVisualContext);
 
}

If you’re not familiar with OpenGL it would help you to know that it’s all about the current state. What does this mean? Well, simply put, it means that the call you are making right now, applies to whatever state the context is in. These two calls are what are the most important for our purposes.

1
2
3

glEnable(CVOpenGLTextureGetTarget(currentFrame));
glBindTexture(CVOpenGLTextureGetTarget(currentFrame), 
                  CVOpenGLTextureGetName(currentFrame));

With these two calls we have told OpenGL which texture to use. Now, every subsequent call applies to this texture until the state is changed to something else. So now, when we set a color or draw a quad, it applies to the texture that has been set here.

Conclusion

I love both of these technologies, QuickTime and OpenGL. They are so powerful. It’s harnessing the power that’s the trick. I’ve got some other ideas for some related posts that I plan to cover in the weeks to come, but this was a real breakthrough for me. With the help of John Clayton, Jean-Daniel Dupas, and David Duncan on the cocoa-dev list, I was able to get the sample code put together for this post. Feel free to ask questions in the comments. I will do my best to answer, but I’m still pretty new to these technologies. Write some code yourself and have fun. This is really exciting stuff. Until next time.

About The Demo Code

John Clayton has some issues getting the code to work on his Mac Pro. I successfully ran it on my MacBook Pro, and the family iMac without any issues. Anyhow, we’re not sure what the problem is, so if you do run into trouble, let me know. Maybe we can figure it out. Meanwhile we’re investigating it as we have time.

Update: John Clayton figured it out. Apparently the visual context needs to be reset because the pixel format is not correct on the first run. We just reset the visual context now in the call to -copyCGLContextForPixelFormat and everything seems happy. The demo code has been updated to reflect the change.

Quicktime CAOpenGLLayer Demo

A Cocoa framework is another project type in Xcode. The end result is a bundle, similar to an application bundle. Inside of this bundle is the compiled code you wrote and any headers that you want exposed. The headers are important. Without them, just like any other piece of Objective-C code, it is very hard to code against.

Building a Framework in Xcode

Building a framework within Xcode is straight-forward. There is already a project template specifically for Cocoa frameworks so that is where we will start.

Once the framework is build, the rest of the code development is almost identical to an application bundle. C, C++ and Objective-C files can be included along with nibs, xibs, images and just about anything else we would stick into an application bundle. When the code is complied, the end result is also a bundle, again very similar to an application. The bundle structure, however, is a bit different and will be discussed in detail below.

Care must be given to the headers that we write and that we expose as part of that framework. In general, we really only want to expose the headers that users of the framework are going to interact with. If a header is completely internal to the framework then do not plan on exposing it. In addition, it is a good idea to have a wrapper header that users can import and gain references to all of the public headers. Since a framework will undoubtably contain more than one class file, how do we roll all of that up into one header? One solution I have seen is to create a separate header file and copy all of the interfaces into it. Unfortunately that solution can become unwieldy very quickly. My preferred solution is to build a “wrapper” header file that imports all of the headers that you want exposed. This is demonstrated in Code Sample 1.

#import "ExampleOne.h"
#import "ExampleTwo.h"

Once the framework is built but before we distribute it, there are a few tweaks we need to deal with. First, we need to declare which headers are public and which are project. There is also an option for private but honestly I cannot see the logic behind them. If we flag a header as private it gets included in the resulting framework in a directory called “PrivateHeaders”. However, if we flag a header as “project” then it is correctly not included in the framework. Public headers are naturally included in the framework under the Headers directory.

Which headers should be included

For some headers, the answer to that question is easy. However, there are some gotchas that need to be planned around. If an object is referenced in one of the public headers then its header