Integrating a Digital Membership Card System with Dataiku for Member Analytics

Rixcy

Hello everyone, I’m exploring a use case where we connect a digital membership card system with Dataiku to improve member analytics and engagement for a large user base (50k+ members). We currently manage membership data through a platform like MembershipAnywhere, which issues Apple Wallet / Google Wallet-based digital membership cards, tracks renewals, and captures member activity such as check-ins, rewards usage, and engagement events.

What we are trying to achieve with Dataiku:

• Centralize membership and engagement data from the digital membership platform
• Build segmentation models (active vs inactive members, frequent visitors, high engagement users)
• Analyze membership renewal behavior and predict churn
• Combine CRM + membership card usage data for a unified view
• Create dashboards for marketing and operations teams

I would like to understand the best approach in Dataiku for:

1. Ingesting data from a third-party membership card system (API / batch sync)
2. Structuring member identity matching across systems (CRM + membership platform)
3. Building scalable pipelines for 50k+ members data
4. Any recommended plugins or patterns for real-time or near real-time syncing

If anyone has worked on similar integration (membership systems, loyalty programs, or wallet-based card platforms), I would really appreciate your guidance or architecture suggestions.

Turribeach

https://community.dataiku.com/discussion/45829/integrating-a-digital-membership-card-system-with-dataiku-for-member-analytics

I don;t' see any issues with your requirements and Dataiku should be a great fit for what you are trying to do. Some pointers in your questions:

1. Ingesting data from a third-party membership card system (API / batch sync)

Dataiku is mostly a Python based runtime engine so if you have a Python API or can access your API via Python easily then you should be good.

2. Structuring member identity matching across systems (CRM + membership platform)

Ingest data from all your systems and consolidate in Dataiku. That's the standard pattern for more projects in Dataiku.

3. Building scalable pipelines for 50k+ members data

While Dataiku allows you to use local file systems or bucket/block cloud storage these are not performant enough to support serious data ingestion and data manipulation. So make sure you pair Dataiku with a cloud-based scalable data platform like Snowflake, Databricks, BigQuery, etc. You may also need to offload compute to Kubernetes if you compute needs are high.

4. Any recommended plugins or patterns for real-time or near real-time syncing

Have a look at the streaming data section of the documentation. Basically there is a type of recipe called Continuous recipe which works with a Continuous activity and is mainly designed for real-time or near real-time data feeds.

tgb417

@Rixcy

I’ve worked with memberships in museums and theatre contexts. Glad to chat with you dm me.

You can do a lot of good work without having to resort to Python in Dataiku. Do look at the API Connect Plugin to Dataiku. You can also do some interesting things with vibe coding with Claude or maybe the very new Cobuild in 14.7.0.

You can try proof of concept work without resort to the more expensive data management solutions that @Turribeach is suggesting.

Turribeach

https://community.dataiku.com/discussion/comment/47549#Comment_47549

@tgb417 Be very careful with Cobuild and non-Code Studio AI Assistants in version 14.7.0 and above. Dataiku has now made it very easy to use Cobuild and non-Code Studio AI Assistants in the "bring your own LLM" mode. This means you can use the LLM mesh to connect to AWS bedrock, OpenAI, etc models using your own credentials and use these models without sending any business data to Dataiku.

However and crucially as per the official documentation "Use of the AI Services involves metered usage. AI Services may include a limited amount of included free usage and/or additional credits subject to fees as set forth in applicable Orders". What this means in practice is that Dataiku is now charging customers to use AI services even if you use and pay for your LLM usage to an external vendor.

Personally I think this monetisation move is a huge mistake. If anything Cobuild was shaping up to be the reason to continue to use Dataiku in this new GenAI world allowing non-technical users (aka clickers) and coders to be able quickly create data flows and models. By adding an unknown cost barrier to these AI features they will make it much harder if not impossible for enterprise customers to use them.

Dataiku really needs to decide how it wants to commercialise their product. Either they can use pay per-user license model or they can move to Pay-As-You-Go (PAYG) / Pay-Per-Use model, but you really can't have it both ways. And they certainly should not change the rules of the game for licensed customers with an existing valid contract.