Announcing YADA

Announcing YADA

https://github.com/Novartis/YADA

It is my great pleasure to inform you that my open source thin-server, data abstraction framework, YADA, has been released. 

YADA is like a Universal Remote Control for data.

For example, what if you could access

any data set
at any data source
in any format
from any environment
using just a URL
with just one-time configuration?
You can with YADA.

Or, what if you could get data

from multiple sources
in different formats,
merging the results
into a single set
on-the-fly
with uniform column names
using just one URL?
You can with YADA.

What is YADA?

• YADA exists to simplify data access and eliminate work.
• YADA is a lightweight framework for data retrieval, searching, storage, and manipulation.
• YADA is an instant web service for your data.
• YADA is a tool to enable efficient development of interfaces and data-processing pipelines.
• YADA is as an implementation of Thin Server Architecture.
• YADA is anti-middleware.
• YADA is an acronym for "Yet Another Data Abstraction."
• YADA is an open source software framework distributed by Novartis Institutes for BioMedical Research under the Apache 2.0 license.

Its raisons d'être are to enable efficient, non-redundent development of data-dependent applications and utilities, data source querying, data analysis, processing pipelines, extract, transform, and load (ETL) processes, etc. YADA does all this while preserving total decoupling between data access and other aspects of application architecture such as user interface.

Still like "Huh?"

YADA is a software framework, which means it is a collection of software tools forming a basic structure underlying a system, for developers and data analysts to use to create new tools and solutions in a new way.

The novelty and utility of YADA lies in its centralization of management of data source access configuration. It simplifies these aspects of software development by eliminating many steps, thereby enabling rapid development, standardization of access methods, and the code in which these methods are implemented. Further it strongly encourages reuse of existing configurations (once configured.)

As a result of these configuration facilities, YADA enables the aggregation or integration of data from multiple data sources using a standard method, agnostic with regard to any vendor or technology-specific details of disparate data source implementations.

For example, the conventional method to access, or furthermore, combine data from say, an Oracle® database, and a web service, is to write code which connects to each database or service independently using different methods and libraries, write code to execute embedded queries independently, also using different methods and libraries, and write code to parse and aggregate the separately acquired data sets. Then the data is typically fed to an analysis tool.

With YADA, the data source connections and application-specific queries are stored securely and centrally, the queries are executed using identical methods (despite the different sources,) and the data can be integrated or aggregated on-the-fly.

For software developers and data analysts alike, these features offer potentially tremendous time savings, faster time-to-delivery, and a larger percentage of time focused not on the tedium of configuration, but on the specific context of a software solution or data analysis.