Deet is a universal data mapper and SQL-Database engine, that implements high-performance and security requirements for large-scale data analysis applications. Deet is based on the Python programming language and actively developed as the database component of the Vivid Code framework.
The primary goal of Deet is to separate data integration and data analysis into independent tasks, by providing a universal data interface for machine learning- and data analysis applications. To achieve this goal, Deet implements the two fundamental layers of a data warehouse:
The integration layer of Deet utilizes SQLAlchemy to allow it's connection to a variety of SQL-Databases (e.g. IBM DB2, Oracle, SAP, MS-SQL, MS-Access, Firebird, Sybase, MySQL, Postgresql, SQLite, etc.). Thereupon it provides native support for flat file databases (e.g. CSV-Tables, R-Table exports), laboratory measurements and data generators.
The staging layer of Deet is implemented as a native SQL-Database engine, featuring a DB-API 2.0 interface with full SQL:2016 support, a vertical data storage manager and real-time encryption. This allows the data analysis application to integrate a variety of different data sources, by using a unified data interface (and SQL dialect).
Installation using PIP
$ pip install deet
Deet or 'How to tame the Plug Jumble': Online analytical processing and predictive analytics in combination with machine learning provides a new challenge in data-warehousing: The response time for large transactions of data from different domains. ...more
Three obstacles in data science and one vision: For the current development- and exploration process in data science three obstacles in particular appear as outstanding hurdles, when it comes up to realize projects - and even more, when it comes up to venture collaborations. ...more