News
Ferry — Database-Supported Program Execution
With project Ferry we try to establish a connection between two somewhat distant shores: programming languages and database technology. Ferry explores how far we can push the idea of relational database engines that directly and seamlessly participate in program evaluation to support the super-fast execution of data-intensive programs written in a variety of (general purpose) programming languages. Ferry builds on technology developed in the context of our project Pathfinder.
Database systems as programming language co-processors
Relational database systems provide the best understood and most carefully engineered query processing infrastructure available today. Notwithstanding these data processing capabilities, RDBMSs are often operated as plain stores that do little more than reproduce stored data items for further processing outside the database host. With Ferry, instead, we aim to turn the database system into an efficient, capable, and highly scalable co-processor for your programming language’s runtime:
Programs that touch and move huge amounts of data benefit the most. You may continue to process your data using your programming language's very own syntax, idioms, and functions — Ferry will compile selected fragments of your code into (sequences of collaborating) database queries for an efficient, possibly parallel, execution on an associated database engine.
Challenges
To this end, we search for, design, and implement new compilation strategies that map data types (e.g., nested and ordered lists, arrays, dictionaries), control structures (e.g., nested iteration, conditionals, variable assignment and reference), and idioms prevalent in functional programming and scripting languages into efficient database queries. Here, we try to push the limits of what has been considered possible (this includes algebraic data types, pattern matching, higher-order functions, and closures, to name a few).
Supported Programming Languages
While the Ferry language is our very own sandbox in which we dig deep and freely, we build on what we find to bring elaborate database support to more widely known languages: our current focus is on LINQ, Ruby, Haskell, Batches for Java and Links, but we consider this list as open-ended.
Target Database Back-ends
Ferry targets off-the-shelf SQL database systems (e.g., IBM's DB2, Microsoft's SQL Server, or PostgreSQL) as well as a number of off-beat database engines (e.g., VectorWise's X100, CWI's MonetDB, or Kx Systems' kdb+).
Demo
This demo video provides a flavor of the ideas behind Ferry:
Publications
- Bringing Back Monad Comprehensions (also available: Slides and Video).
George Giorgidze, Torsten Grust, Nils Schweinsberg, and Jeroen Weijers.
In Proceedings of the ACM SIGPLAN Haskell Symposium (Haskell 2011), Tokyo, Japan. ACM, 2011.
- Haskell Boards the Ferry: Database-Supported Program Execution for Haskell (also available: Slides).
George Giorgidze, Torsten Grust, Tom Schreiber, Jeroen Weijers.
Proceedings of the 22nd Symposium on Implementation and Application of Functional Languages (IFL 2010), Alphen aan den Rijn, Netherlands, September 2010, To be published by Springer LNCS (to appear in 2011), Best Paper Award.
- Avalanche-Safe LINQ Compilation (also available: Slides).
Torsten Grust, Jan Rittinger, Tom Schreiber.
Proceedings of the VLDB Endowment, Volume 3, September 2010 (VLDB 2010), Singapore, September 2010.
- Thirteen New Players in the Team: A Ferry-based LINQ to SQL Provider (also available: Poster).
Tom Schreiber, Simone Bonetti, Torsten Grust, Manuel Mayr, Jan Rittinger.
Proceedings of the VLDB Endowment, Volume 3, September 2010 (VLDB 2010), Singapore, September 2010.
- Ferry: Database-Supported Program Execution (also available: Poster and Video, MP4, ca. 10 MB).
Torsten Grust, Manuel Mayr, Jan Rittinger, Tom Schreiber.
Proceedings of the 28th ACM SIGMOD Int'l Conference on Management of Data (SIGMOD 2009), Providence, Rhode Island (USA), June 2009.
Talks
Team
|
