Quote:
Originally Posted by suzzer99
There's a batch process every night or so that saves user changes to the CRM. It actually has to be reviewed by humans so they don't change their name to Mickey Mouse or something. Handling that will be a whole other ball of wax. But the team already tried to tackle it once in the past (on their first attempt at this app) so a lot of the business logic has at least been captured and there's old code to look at.
The data which will actually be saved in our app DB will be extra stuff that isn't stored in CRM. Picture a CRM that stores user vital information like address, phone, birthday. But our app stores user preferences - like which page they'd like to view first - and the all important mapping of Cognito user ID to CRM user id. And probably some other stuff we haven't thought of yet.
As far as schema changes - the app will need to be smart enough to treat missing properties as the default value. The way I've done this in the past is to deep merge the user object over the default object, then evaluate that.
Yes only lookup user info by primary key - the User ID supplied by Cognito. The app will never make queries across all users.
Not sure experienced with NoSql but this feels like a good application of it to me.
It's a good point though that I need to decide where to cache the bulk of the user data, which comes from the CRM. Mostly it will be read-only for the app. But we'll need to have it available to every request. Something like DAX - which is a hot caching layer over dynamodb - looks promising. However that means I'll have to save the CRM user data in dynamo - so it can be hot-cached - and then worry about how to invalidate if it gets out of sync. Hmmm
Got it. So this is still basically core application data, it just happens that you own this data, and you treat CRM as an external service for some user data. If your data is simple and it's always queried by id/key then noSQL should be fine. Just keep in mind that relational databases give you a lot of stuff for free like validations, data integrity, schema management, failover/master promotions etc. With noSQL systems you end up writing a lot of this stuff in application code over time.
There's also the option of incorporating the CRM data into your model, as you alluded to with the caching idea. But can go further and actually save the CRM data as part of your model in the database. There are ways to sync data from another system, such as pub/sub or callbacks(web hooks) with retries (prob need a queue to do callbacks reliably).