Saving JSON to Core Data
Hi, I’m new here. You may know me as @atomicbird on Twitter. Just a few days ago my book Core Data for iOS: Developing Data-Driven Applications for the iPad, iPhone, and iPod touch (co-written with the excellent Tim Isted) was published, and Matt invited me to contribute some Core Data tips to CIMGF. I’m going to start off discussing taking JSON data from a web service and converting it to Core Data storage. Along the way I’ll cover how to inspect managed objects to find out what attributes they have and what the attribute types are.
Publishing lead times being what they are, this post covers information not included in the book.
The Ugly/Crude Way
I’m going to assume that the incoming JSON more or less matches your managed object, i.e. that you have a JSON dictionary with keys that match the attribute names of your Core Data entities. Given a dictionary jsonDict that has been created by parsing incoming JSON data using a JSON library (TouchJSON and json-framework are good choices) and a managed object imaginatively called myObj, the dead simple approach is to just do something like:
[myObj setValue:[jsonDict objectForKey:@"name"] forKey:@"name"];
[myObj setValue:[jsonDict objectForKey:@"city"] forKey:@"city"];
[myObj setValue:[jsonDict objectForKey:@"phone"] forKey:@"phone"];
…and on and on and on ad nauseum. It works but it’s astoundingly ugly. It also makes maintenance more challenging, since any changes to the data structure require changes not only to your Core Data model and related managed object classes but also to your data import code.
The Easy/Dangerous Way
Cocoa provides a method that makes this much, much simpler. Thanks to key-value coding there’s a method in NSObject that reduces the above to a single line:
[myObj setValuesForKeysWithDictionary:jsonDict];
This will run through key/value pairs in jsonDict and set the same key/value pairs on myObj. The only caveat is that the key names must match, but that shouldn’t be a problem assuming you’ve named them the same in your data model.
Actually there’s one more caveat, and it’s a biggie. In this method, the keys contained in the dictionary determine what keys are used on the managed object. But what if the dictionary contains a key that’s not part of the entity description? Then your app crashes with an error about how the object is not key-value coding compliant for the key. Using this method here puts your app’s stability at the mercy of the web service you’re using. If you don’t have absolute control over the web service, you’re running serious risks using this method here. We can do better.
Inspecting Managed Objects
Since we’re using Core Data, we can use NSEntityDescription to inspect a managed object’s attributes and turn the logic above around. Instead of using the incoming dictionary to determine what key/value pairs to use, we can use the managed object.
You can look up the entity description for a managed object by asking for its entity. That gives you an NSEntityDescription which you can then ask for all kinds of useful information, like what attributes exist. That leads to a safer approach when converting from JSON:
NSDictionary *attributes = [[myObj entity] attributesByName];
for (NSString *attribute in attributes) {
id value = [jsonDict objectForKey:attribute];
if (value == nil) {
continue;
}
[myObj setValue:value forKey:attribute];
}
This code iterates over the entity’s attributes, looks up a key in the dictionary for each of them, and applies that key/value pair to the managed object. That’s still nice and generic, and has the advantage that changes to incoming data won’t crash the app any more. It looks up values in the dictionary only for attributes that actually exist in the entity description. Extra dictionary keys are simply ignored.
The check for nil is there because the code does hit all of the entity’s attributes. If the entity has any attributes that aren’t present in the incoming data (a “favorites” flag, for example) the code would end up setting a nil value for the attribute. That could lead to accidentally wiping out data that you really want to keep.
Handling Broken and Inconsistent JSON
The JSON standard is quite clear about how to distinguish strings from numbers– basically, strings are surrounded by quotes and numbers are not. JSON web services however, are not always good about following this requirement. And even when they are, they are not always consistent from one record to another.
An example, similar to one I encountered recently: Size information for clothing. Sizes are provided by the web service and are typically something like “30-32”, “34-36”, etc. These are correctly quoted as strings in the JSON, and the app saves them as string attributes and displays them to the user as is.
But sometimes the size just has one number, e.g. “8”, “10”, etc. In this case the server drops the quotes, making them numbers. My JSON parser correctly produces an NSNumber. Only I want to save this in my entity’s string attribute! I can use Objective-C introspection to see if I received an NSString or an NSNumber from my JSON parser, but I also need to know what type the managed object expects for the property. I briefly considered breaking my JSON parser so that it would always return NSString, so at least I would know what to expect from it. Fortunately NSEntityDescription came to the rescue again.
Besides asking the entity description what its attribute names are, you can also inquire about the attribute types configured in the Core Data model. This is returned as an NSAttributeType. Using this, we can expand the code above to handle mismatched data types. It’s probably a good idea to abstract the code for easy reuse, too, so I’ll put it in a category on NSManagedObject:
@implementation NSManagedObject (safeSetValuesKeysWithDictionary)
- (void)safeSetValuesForKeysWithDictionary:(NSDictionary *)keyedValues
{
NSDictionary *attributes = [[self entity] attributesByName];
for (NSString *attribute in attributes) {
id value = [keyedValues objectForKey:attribute];
if (value == nil) {
// Don't attempt to set nil, or you'll overwite values in self that aren't present in keyedValues
continue;
}
NSAttributeType attributeType = [[attributes objectForKey:attribute] attributeType];
if ((attributeType == NSStringAttributeType) && ([value isKindOfClass:[NSNumber class]])) {
value = [value stringValue];
} else if (((attributeType == NSInteger16AttributeType) || (attributeType == NSInteger32AttributeType) || (attributeType == NSInteger64AttributeType) || (attributeType == NSBooleanAttributeType)) && ([value isKindOfClass:[NSString class]])) {
value = [NSNumber numberWithInteger:[value integerValue]];
} else if ((attributeType == NSFloatAttributeType) && ([value isKindOfClass:[NSString class]])) {
value = [NSNumber numberWithDouble:[value doubleValue]];
}
[self setValue:value forKey:attribute];
}
}
@end
This code inspects the types of both the value found in the dictionary we created from the incoming JSON and the attribute on the managed object, and if there’s a string/number mismatch it modifies the value to make sure it matches what’s expected.
This is getting pretty useful. Not only is it a generic JSON-to-NSManagedObject conversion, it also handles mismatches between numeric and string types without needing any entity-specific information. It could be even better, though.
Handling Dates
JSON does not have a date type, but dates are nevertheless common in JSON. They’re just represented as strings using one date format or another. Only you probably want an NSDate, not a string. NSDateFormatter is really useful here, but wouldn’t it be nice to make the date conversion generic as well, so you don’t need to special-case your date attribute? Can you see where I’m going with this?
Having written the category method above, it’s not much more work to add an optional NSDateFormatter argument, and then to use it whenever an entity’s attribute expects a date. This modified version adds that argument, and an extra case to the “else … if” chain:
@implementation NSManagedObject (safeSetValuesKeysWithDictionary)
- (void)safeSetValuesForKeysWithDictionary:(NSDictionary *)keyedValues dateFormatter:(NSDateFormatter *)dateFormatter
{
NSDictionary *attributes = [[self entity] attributesByName];
for (NSString *attribute in attributes) {
id value = [keyedValues objectForKey:attribute];
if (value == nil) {
continue;
}
NSAttributeType attributeType = [[attributes objectForKey:attribute] attributeType];
if ((attributeType == NSStringAttributeType) && ([value isKindOfClass:[NSNumber class]])) {
value = [value stringValue];
} else if (((attributeType == NSInteger16AttributeType) || (attributeType == NSInteger32AttributeType) || (attributeType == NSInteger64AttributeType) || (attributeType == NSBooleanAttributeType)) && ([value isKindOfClass:[NSString class]])) {
value = [NSNumber numberWithInteger:[value integerValue]];
} else if ((attributeType == NSFloatAttributeType) && ([value isKindOfClass:[NSString class]])) {
value = [NSNumber numberWithDouble:[value doubleValue]];
} else if ((attributeType == NSDateAttributeType) && ([value isKindOfClass:[NSString class]]) && (dateFormatter != nil)) {
value = [dateFormatter dateFromString:value];
}
[self setValue:value forKey:attribute];
}
}
@end
Conclusion
Saving JSON data to a managed object is one of those things that’s not as easy as it seems at first glance. Making it happen is easy enough, but making it happen safely in maintainable code can quickly get complicated. Fortunately, Core Data has your back and will help you work out what needs to happen along the way.
This is pretty much the same thing I came up with mine, though I added a slight twist to mine. You can add custom keys to the userInfo dictionary for attributes in your object model, allowing you to customize the keyPath within the JSON data that you want to retrieve. This means the key name in Core Data can be different or can be the result of a deeply-nested property in the JSON data.
Doing it this way also lets you specify a custom selector for doing special formatting / processing of the incoming value, in case you need to do something more than dateFromString: or numberWithInteger, etc.
This post is a good primer, but it gets a lot more complex in practice. Handling relationships, implementing create-or-update, and dealing with NSNull values/other parser/API quirks are just a few common scenarios that come to mind. It’s also worth mentioning that there are a number of projects working on a generalized solution for this task. RestKit (http://github.com/twotoasters/RestKit) seems to have the most activity at the moment.
@NachoMan, passing in a userInfo dictionary is a really good idea, it takes what I was doing with the date formatter and further generalizes it. With a set of known userInfo keys it would allow for all kinds of customization.
@Cameron, Sure, this isn’t everything, the motivation here was to replace setValuesForKeysWithDictionary with something more robust and safe, without getting into the other steps that are frequently necessary.
Thanks! I cannot wait for your next post, entitled “How I worked around the changes in Mac OS X 10.7 and made MondoMouse fully compatible with Lion from DR 3 onwards.”
Looking forward to it! (^_^)
Awesome post, did that several months ago quite similarly! Nice work. I like the way you handle the KVC, one thing I’d suggest though:
A lot of web services / JSON data contains “NULL” values, as they map database entries with the null option enabled directly to null, i.e. : { edited_on : NULL } if there was never set a value..
in that case you’d add standard values to needed properties or skip the KVC process entirely, i.e:
else if ((attributeType == NSDateAttributeType) && ([value isKindOfClass:[NSNull class]]) && (dateFormatter != nil)) {
value = [dateFormatter dateFromString:@”2010-01-01 12:00:00″];
}
Cheers.