Country: United States
FINRA is a not-for-profit organization dedicated to investor protection and market integrity. It regulates one critical part of the securities industry — brokerage firms doing business with the public in the United States. FINRA, overseen by the SEC, writes rules, examines for and enforces compliance with FINRA rules and federal securities laws, registers broker-dealer personnel and offers them education and training, and informs the investing public. In addition, FINRA provides surveillance and other regulatory services for equities and options markets, as well as trade reporting and other industry utilities. FINRA also administers a dispute resolution forum for investors and brokerage firms and their registered employees.
We use innovative AI and machine learning technologies to keep a close eye on the market and provide essential support to investors, regulators, policymakers and other stakeholders.
The Financial Industry Regulatory Authority (FINRA) is a not-for-profit organization authorized by the U.S. Congress to protect investors and ensure market integrity through effective and efficient regulation of broker-dealers. It writes and enforces rules governing the activities of more than 3,400 broker-dealers representing more than 630,000 brokers, examines firms for compliance, fosters market transparency, and educates investors.
Every day, FINRA oversees up to 600 billion market events, including equities, options and fixed income products in the United States. Petabytes of historical market data needs to be analyzed to uncover insider trading and other strategies used to gain an unfair advantage. This constitutes a significant volume of data that requires considerable computing power for effective analysis.
FINRA’s mission is to protect investors and promote market integrity. To achieve this, it is important to quickly respond to market events, especially in today’s highly volatile market.
FINRA needs to look at the data to answer questions like: "Are there market events/participants that are threatening market integrity?" or "Are there regulatory responses required for these events to fulfill our mission?", and more.
This means efficiently analyzing hundreds of billions of market event data across hundreds of sources. This efficiency is critical when we need to react within days or even hours.
We therefore need to leverage interactive data analytical solutions to respond to critical and potentially harmful market events. Due to highly complex nature of our work, we require close collaboration across examination, surveillance, enforcement, and technology functions.
Our goal is to have an integrated platform to data analytics that avoids duplication, translates data into real results and supports coordinated contributions from all the various groups engaged in this activity.
We chose Dataiku as the Data Analytics/Data science platform for our users. The aim for this project is to enable and empower our users with platforms and solutions to derive better insights from large volume of data so that they can identify and stop bad actors faster.
Dataiku was leveraged by diverse user personas as listed below and they were all able to come together and collaborate in a single and shared platform. It enabled real-time interactive insights generation across the organization.
Within a single platform we can analyze data sets of various sizes pulled from multiple different data sources. Between Dataiku’s architecture and Spark’s scalability, we essentially have no data size limits.
Dataiku has enabled a new type of data analysis paradigm, the End User Computing applications (EUCA). The EUCA is essentially an ability for knowledge workers without technical skills to access and analyze data regardless of its size (petabytes!) and the complexity of the analysis (Spark transformations, stats, ML, whatever).
FINRA can build the EUCAs using the following three Dataiku capabilities. First, the no-code environment and visual recipes for defining the transformations. Any non-tech person who mastered Excel can use Dataiku’s visual recipes and start getting insights from the data very quickly. Secondly, Dataiku provides non-coders and Data Scientists with the capability to develop web-based applications using just a few mouse clicks or a short Python program. What could take months to develop can now be done in days. Third, Dataiku’s flexible WebApp features have enabled FINRA to stand up custom Dash-based apps that combine large datasets of pre-processed market data with user-generated inputs to assess risks at member firms. The flexibility of Dash-based WebApps have allowed small teams of data scientists to partner with business SMEs to produce prototype tools in a fraction of the time it would take to develop in traditional development platforms.
We have found that the EUCAs are the fastest and most scalable way to perform ad hoc analyses. Moreover, it is perfect for trying new ideas and innovations.
Dataiku has not only made data more accessible but also empowered business analysts, non-coders to harness the power of large datasets that were previously out of our reach. Analysts are now autonomous in analyzing data for their own needs, which greatly improves our time-to-insight and enables us to react faster as an organization.
Business Area Enhanced: Analytics
Use Case Stage: In Production
Leveraging Dataiku for self-service analytics had had a broad impact:
In summary, we are now conducting more analyses per year, making it times faster, with the better quality, and spending less money for technology human and compute resources.
As an end-to-end solution, many capabilities of Dataiku helped FINRA become more data-driven and to shift towards End User Computing. The following Dataiku’s capabilities were most crucial for that:
In addition, we recognize a very high value of these Dataiku components:
Value Range: Millions of $