In 2009, NoSQL introduced a new category of databases. They were highly specialised and initially seemed to herald the end of the dominance, of relational SQL databases, which had previously dominated the market almost unchallenged and lacked the specialisation capabilities to cover different needs and data types. But today, general-purpose platforms are making a comeback.
It is no longer just companies from the software industry that depend on the rapid development of innovative applications for the success of their business models. Whether airline, energy supplier, retailer, or fitness provider: A highly functional custom app is standard today and is expected by customers. Frequently used apps such as social media or streaming platforms define the expectation of responsiveness, optimization for mobile devices, preparation of desired content and information, security, personalization, and updates and insights in real-time. The “engine” behind these applications, which have long been part of our everyday lives, is data and their storage and processing in databases.
The forerunner of the Structured Query Language (SQL), on which relational data platforms are based, was already developed in the 1970s. SQL databases have long been the basis for data-driven applications. However, the increase in mobile devices and apps placed new demands on the agility of databases. The data collected for these applications is not limited to a limited number of attributes. The data sets have become increasingly unstructured and complex. The rigid structure of relational databases, consisting of rows and columns, does not solve this complexity and makes storing, analysing, and querying difficult.
The relational model reached its limits in the noughties. Ad hoc changes in the data structure to be stored or in their query patterns remained difficult. For example, architectures such as Large online retailers, with their massive amounts of data and their need for back-end data analytics, drove demand for a different approach.
Specialised Models: Competing With But Not Replacing SQL
In the wake of the NoSQL 2009 event, various new database types emerged: document databases, graph databases, columnar databases, in-memory databases, key-value stores, streaming platforms. All specialised solutions developed for specific use cases were initially only suitable for these. They all had particular advantages and disadvantages.
Relational databases, meanwhile, have not disappeared from the scene. Many companies still use them today, but their weaknesses slow down developers when it comes to querying, analysing, and effectively processing the ever-growing flood of data. Of the new models, document databases are the most common alternative today. Its popularity is due to its flexibility, allowing many data structures to be represented. Objects are stored as documents with possibly different attributes. With the help of deep learning, these attributes or “tags” increasingly enable pattern recognition, which speeds up the finding of results in queries.
Document Databases: Most Popular SQL Alternative Among Developers
Instead of the relational model’s rigid row and column structure, document databases map their internal representation more directly to objects in code, eliminating the additional layer of mapping required by relational databases. The benefit of this similarity is that since they are based on JSON-like documents, they are incredibly intuitive for developers to use. JSON is a language-independent and human-readable format that has become a widely used standard for storing and exchanging data, especially web applications.
A significant advantage of document databases is that the structure of the data, the so-called schema, does not have to be predefined in the database and that ad hoc changes in this schema are possible at any time. Such changes can occur for a variety of reasons:
- Source data is being delivered in a different format.
- Newly enhanced information is being collected.
- Recent queries are being sought to be supported.
Schema management is an essential part of working with relational databases, where the structure of the data must be pre-mapped to support queries about the relationship between different elements. The ability to support these changes without extensive database and application code rework increases the flexibility of developers.
Which Approach Does The Future Belong To?
The document database offers excellent potential for companies to use big data sensibly. Its vertical and horizontal scalability ensures that it “grows” with developing a business model and the growth in data volumes. However, relational databases are still common in web-based applications and online publishing. Specialised models are selected depending on the purpose and requirements. While it seemed that the future belonged to different, specialised databases, the trend is now reversing, as many companies are moving back towards the “generalist” models suitable for various use cases throughout the organisation.
There are multiple reasons for this:
- Users of specialised databases are pushing for feature enhancements, so they don’t have to switch between different data stores because they want to consolidate existing datasets across organisational and technological boundaries.
- Developers also want more integration. Nobody likes the relational structure as the only option back. However, instead of the effort involved in querying and analysing data within this structure, they are increasingly faced with the effort involved in integrating various specialised databases, each of which underlies specific applications. In extreme cases, this can mean that the higher integration effort nullifies the advantages of your agility in a particular area.
The Trend Towards Multi-Purpose Data Platform
Database providers have long recognized the trend reversal and are reacting: Their models are now much less highly specialised than in the early days of the NoSQL concept. MongoDB’s document database has provided data lake, charting, and other analytics capabilities since version 4.2 and time series capabilities since version 5.0 (2021). Different models are also expanding their solutions with new functions, thus moving more towards universal applicability.
This consolidation also has tangible economic reasons. The operating costs that arise from the maintenance of several specialised data stores are considerable and often lead to the emergence of operational silos that no company can use in the process of digitization: data that is stored in a particular format in a system must be converted, so they can be shared with other systems and the teams that use them.
Increased Safety Standards Are Easier To Meet
Data security and data protection are very high on companies’ agendas: the requirements for handling sensitive data have become increasingly strict in recent years, the increased number of mobile devices and access points due to remote work and the increasing shift of enormous amounts of data to the cloud has led to cyberattacks skyrocket with data theft. The higher the number of platforms, the more complex it is to meet compliance and security requirements. Each datastore must be secured separately. Its configuration must be checked for compliance with internal and external regulations. A multi-purpose data platform, on the other hand, provides a single point of control for security and compliance, in addition to operational simplification.
While specialised platform vendors add functionality to their databases to broaden their suitability for different use cases, general-purpose platform vendors add technical functionality simultaneously. This also increasingly eliminates the competitive advantage of specialised data storage and thus the need for complex maintenance. Modern application data platforms can support text search, time-series data processing, and analytics directly in the core platform without the need to integrate and set up data exchange with particular systems. While database vendors increasingly moved away from general-purpose workloads following NoSQL in 2009, the reversal has been evident over the past three years.
Conclusion
When selecting a single database, organisations should compare traditional relational data models and newer approaches. Instead, they should consider a comprehensive data platform whose underlying data model can support as many requirements as possible. The transformation in the database market is being driven by the demands of developers who need to meet the demands of end-users who want and need to derive business-relevant insights from their data.
The race winners will be data platforms that enable the delivery of desired functions. By definition, this goal requires a unified view of the data and various tools that can be used to analyse it. This is what modern application data platforms offer. So the future will likely belong to those solutions that combine the best of both worlds and can be used for a wide range of applications and provide specialised functions when required.
ALSO READ: Samsung Galaxy S22 Ultra In The Test: Or Is It A Galaxy Note?
