Feed: MemSQL Blog.
Author: Floyd Smith.
One of MemSQL’s strengths is speeding up analytics, often replacing NoSQL databases to provide faster performance. Kurtosys, a market-leading digital experience platform for the financial services industry, uses MemSQL exclusively, gaining far faster performance and easier management across transactions and analytics.
Kurtosys is a leader in the Digital Experience category, with the first truly SaaS platform for the financial services industry. In pursuing its goals, Kurtosys became an early adopter of MemSQL. Today, MemSQL is helping to power Kurtosys’ growth.
Stephen Perry, head of data at Kurtosys, summed up the first round of efforts in a blog post several years ago, titled Why Would I Store My Data In More Than One Database? (Among his accomplishments, Steve is one of the first MemSQL-certified developers.)
In the following blog post, we describe how usage of MemSQL has progressed at Kurtosys. In the first round, Kurtosys had difficulties with their original platform, using Couchbase. They moved to MemSQL, achieving numerous benefits.
Further customer requests, and the emergence of new features in MemSQL, opened the door for Kurtosys to create a new platform, which is used by Kurtosys customers to revolutionize the way they deliver outstanding digital and document experiences to their sales teams and to their external communities of clients and prospects. In this new platform, MemSQL is the database of record.
At Kurtosys, Infrastructure Powers Growth
Kurtosys has taken on a challenging task: hosting complex financial data, documents, websites, and content for the financial services industry. Kurtosys customers use the Kurtosys platform for their own customer data, as well as for their sales and marketing efforts.
The customer list for Kurtosys features many top tier firms, including Bank of America, the Bank of Montreal, Generali Investments, and T. Rowe Price. Kurtosys’ customers require high performance and high levels of security.
Customer focus on security is greater in financial services than in most other business segments. A single breach – even a potential breach that is reported, but never actually exploited – can cause severe financial and reputational damage to a company. So customers hold technology suppliers such as Kurtosys to very high standards.
Alongside security, performance is another critical element. Financial services companies claim performance advantages to gain new customers, so suppliers have to deliver reliably and at top speed. And, since financial services companies also differentiate themselves on customer service, they require suppliers to provide excellent customer service in turn.
(Like Kurtosys, MemSQL is well-versed in these challenges. Financial services is perhaps our leading market segment, with half of the top 10 US financial services firms being MemSQL customers.)
With all of these strict requirements, for financial services companies to trust an external provider to host their content – including such crucial content as customer financial data – is a major step. Yet, Kurtosys has met the challenge and is growing quickly.
“Our unique selling proposition is based around the creative use of new and unique technology,” says Steve. “We’ve progressed so far that our original internal platform with MemSQL, which we launched four years ago, is now a legacy product. Our current platform employs a very modern approach to storing data. We are using MemSQL as the primary database for the Kurtosys platform.”
Kurtosys Chooses Infrastructure for Growth
Kurtosys is adept at innovating its infrastructure to power services for demanding customers. For instance, several years ago, Kurtosys used SQL Server to execute transactions and Couchbase as a high-performance, scalable, read-only cache for analytics.
Initially, the combination made sense. Customers of Kurtosys wanted to see the company executing transactions on a database that’s among a handful of well-established transactional databases. SQL Server fit the bill.
However, like other traditional relational databases, SQL Server is, at its core, limited by its dependence on a single core update process. This dependency prevents SQL Server, and other traditional relational databases, from being able to scale out across multiple, affordable servers.
This means that the single machine running SQL Server is usually fully occupied with transaction processing and would struggle to meet Kurtosys’ requirements, such as the need for ad-hoc queries against both structured and semi-structured data. That left Kurtosys needing to copy data to another system, initially Couchbase, and run analytics off that – the usual logic for purchasing a data warehouse or an operational analytics database.
Couchbase seemed to be a logical choice. It’s considered a leading NoSQL database, and is often compared to other well-known NoSQL offerings such as Apache Cassandra, Apache HBase, CouchDB, MongoDB, and Redis. Couchbase tells its target audience that it offers developers the opportunity to “build brilliant customer experiences.”
NoSQL databases have the ability to scale out that traditional relational databases lack. However, NoSQL databases face fundamental limitations in delivering on promises such as those made by CouchBase. NoSQL databases favor unstructured or less-structured data. As the name implies, they don’t support SQL.
Users of these databases don’t benefit from decades of research and experience in performing complex operations on structured and, increasingly, semi-structured data using SQL. With no SQL support, Couchbase can be difficult to work with, and requires people to learn new skills.
Running against unstructured data and semi-structured JSON data, and without the benefit of SQL, Kurtosys found it challenging to come up with an efficient query pattern that worked across different data sets.
Kurtosys Moves to MemSQL to Power Fast Analytics
As a big data database, Couchbase is an excellent tool for data scientists running analytics projects. However, for day in and day out analytics use, it was difficult to write queries, and query performance was subpar. Couchbase was not as well suited for the workloads and high degree of concurrency – that is, large numbers of simultaneous users – required for internal user and customer analytics support, including ad hoc SQL queries, business intelligence (BI) tools, and app support.
At the same time, Kurtosys needed to stay on SQL Server for transactions. Kurtosys had invested a lot in SQL Server-specific stored procedures. Its customers also liked the fact that Kurtosys uses one of the top few best-known relational databases for transactions.
So, after much research, Kurtosys selected a fully distributed database which, at the time, ran in-memory: MemSQL. Because MemSQL is also a true relational database, and supports the MySQL wire protocol, Kurtosys was able to use the change data capture (CDC) process built into SQL Server to keep MemSQL’s copy of the data up to date. MemSQL received updates a few seconds after each transaction completed in SQL Server. Queries then ran against MemSQL, allowing both updates and queries to run fast against the respective databases.
CDC moved updates to MemSQL, which supported queries.
SQL Server was now fully dedicated to transaction support, with the CDC process imposing little overhead on processing. And, because of MemSQL’s speed, the database was able to easily keep up with the large and growing transaction volume going through the Kurtosys platform.
Kurtosys summed up its approach at the time in a slide deck that’s available within a Kurtosys blog post. The key summary slide is below.
MemSQL-Based Platform Powers New Applications
Kurtosys has now created a new internal platform. One of the key capabilities in the new platform is support for JSON data. In a recent MemSQL release, MemSQL 6.7, JSON data support is a core feature. In fact, comparing JSON data to fully structured data, “performance is about the same, which is a testament to MemSQL,” says Steve. With this capability, Kurtosys can keep many of the same data structures that it had previously used in Couchbase, but with outstanding performance.
Also, when Kurtosys first adopted MemSQL, several years ago, MemSQL was largely used as an in-memory database. This gave truly breakthrough performance, but with accompanying higher costs. Today, MemSQL flexibly supports both rowstore tables in memory and disk-based columnstore. “Performance,” says Steve, “is almost too good to believe.”
The new platform runs MemSQL for both transactions and queries. In the new platform, there’s no longer a need for CDC. Kurtosys runs MemSQL as a transactional database, handling both transactions and analytics.
The new internal platform powers Kurtosys applications with thousands of concurrent users, accessing hundreds of gigabytes of data, and with a database growing by several gigabytes of data a day.
Kurtosys is looking forward to using the new features of MemSQL to power the growth of their platform. As Steve Perry says, in a separate blog post, “What they do, they do right… we use MemSQL to improve the performance of query response.”
Stepping Ahead with MemSQL
MemSQL is a fundamental component of the key value proposition that Kurtosys offers its customers – and cutting-edge platforms, like the one being developed at Kurtosys today, will continue to push MemSQL forward.
To see the benefits of MemSQL for yourself, you can try MemSQL today for free. Or, contact us to speak with a technical professional who can describe how MemSQL can help you achieve your goals.