In-Kernel Berkeley DB Databases and Transactional Storage

We have implemented several prototype transactional file systems (EXPCS 2007, ACM TOS 2008, and an FSL Tech Report in 2008). The most optimal design is one that utilizes an approach that requires minimal changes to the kernel, yet provides the full range of functionality expected of a transactional file system. This work was published in FAST 2009. We explore the impact of a transactional file system on other existing or potential components of the operating system. We have looked at a provenance aware storage system that utilizes a write-optimized logging approach published in TAPP 2009. Currently we are exploring grey-boxing techniques to better define and guarantee recovery semantics for user-land database implementations. We are also exploring the use of a cache-oblivious file system implementation so that a more scalable design for new storage technologies (i.e., flash, phase changing RAM, etc...) are more easily inserted into the operating system's caching mechanism.

Download software.

Journal Articles:

# Title (click for html version) Formats Published In Date Comments
1 Don't Thrash: How to Cache Your Hash on Flash PDF BibTeX The Proceedings of the VLDB Endowment (PVLDB) Aug 2012  

Conference and Workshop Papers:
# Title (click for html version) Formats Published In Date Comments
1 Multi-Tier Caching PDF BibTeX U.S. Patent 9,959,279

 May 2018 issued patent
2 Multi-Tier Caching PDF BibTeX U.S. Patent 9,355,109

 May 2016 issued patent
3 Terra Incognita: On the Practicality of User-Space File Systems PDF BibTeX 7th USENIX Workshop in Hot Topics in Storage and File Systems (HotStorage 2015) Jul 2015  
4 Building Workload-Independent Storage with VT-Trees PDF BibTeX 11th USENIX Conference on File and Storage Technologies (FAST 2013) Feb 2013  
5 Don't Thrash: How to Cache Your Hash on Flash PDF BibTeX 38th International Conference on Very Large Data Bases (VLDB '12) Aug 2012  
6 Multi-Tier Caching PDF BibTeX US Patent Application 13/159,039

 Mar 2012 US Patent Application 13/159,039 publication
7 Don't Thrash: How to Cache your Hash on Flash PS PDF BibTeX 3rd USENIX Workshop in Hot Topics in Storage and File Systems (HotStorage 2011) Jun 2011  
8 Exporting Kernel Page Caching for Efficient User-Level I/O PS PDF BibTeX 26th IEEE Conference on Mass Storage Systems and Technologies (MSST 2010) May 2010  
9 Story Book: An Efficient Extensible Provenance Framework PS PDF BibTeX 1st Workshop on the Theory and Practice of Provenance (TAPP 2009) Feb 2009  
10 Enabling Transactional File Access via Lightweight Kernel Extensions PS PDF BibTeX Seventh USENIX Conference on File and Storage Technologies (FAST 2009) Feb 2009  
11 Rapid File System Development Using ptrace PS PDF BibTeX Proceedings of the Workshop on Experimental Computer Science (EXPCS 2007), in conjunction with ACM FCRC Jun 2007  
12 I3FS: An In-Kernel Integrity Checker and Intrusion Detection File System PS PDF BibTeX Usenix Large Installation Systems Administration Conference (LISA 18) Nov 2004  

Technical Reports:
# Title (click for html version) Formats Published In Date Comments
1 From Tuples to Files: a Fast Transactional System Store and File System PDF BibTeX Stony Brook U. CS TechReport FSL-12-03 May 2012 M.S. Thesis
2 Efficient, Scalable, and Versatile Application and System Transaction Management for Direct Storage Layers PDF BibTeX Stony Brook U. CS TechReport FSL-12-02 Jan 2012 Ph.D. Dissertation
3 SAMT Performace Proof PS PDF BibTeX Stony Brook U. CS TechReport FSL-09-03 Jun 2009  
4 Protecting Control Files With File System Transactions PS PDF BibTeX Stony Brook U. CS TechReport FSL-08-05 Aug 2008  
5 Extending ACID Semantics to the File System via ptrace PS PDF BibTeX Stony Brook U. CS TechReport FSL-06-04 May 2006 Ph.D. Thesis
6 File System Extensibility and Reliability Using an in-Kernel Database PS PDF BibTeX Stony Brook U. CS TechReport FSL-04-06 Dec 2004 M.S. Thesis
7 Enhancing File System Integrity Through Checksums PS PDF BibTeX Stony Brook U. CS TechReport FSL-04-04 May 2004  

Past Students:

# Name (click for home page) Program Period Current Location
1 Nikolai Joukov PhD Jan 2004 - Dec 2006 Research Staff Member, Storage and Data Services Research group, IBM T. J. Watson Research Center (Hawthorne, NY)
2 Gopalan Sivathanu PhD Sep 2003 - May 2008 Software Engineer, Systems Infrastructure group, Google (Mountain View, CA)
3 Rick Spillane PhD Jan 2008 - Feb 2012 Member of Technical Staff, Vmware Inc. (Palo Alto, CA)
4 Avishay Traeger PhD Sep 2003 - Aug 2008 Senior Principal Software Engineer, Red Hat (Raanana, Israel)
5 Charles P. Wright PhD May 2003 - May 2006 Partner, Senior Software Architect, Illumon (New York, NY)
6 Binesh Andrews MS Jan 2012 - Dec 2012 Software Engineer, Infrastructure Storage - Logs at Google (Mountain View, CA)
7 Shrikar Archak MS Sep 2009 - Dec 2010 Senior Software Engineer, Core Deduplication Filesystem group, Datadomain/EMC
8 Saumitra Bhanage MS Sep 2009 - Dec 2010 Technical Yahoo! Software System Development Engineer, Messenger Server group, core team, Yahoo! (Sunnyvale, CA)
9 Manjunath Chinni MS Jan 2008 - Dec 2008 Member of Technical Staff, Engineering Development, Rubrik (Palo Alto, CA)
10 Jay Pradip Dave MS May 2003 - Dec 2003 Head of Product Management, Enterprise platform and Administrative experience, Qualtrics Qualtrics (Seattle, WA)
11 Sagar Dixit MS Sep 2009 - Dec 2010 Senior Member of Technical Staff, Springpath, Inc. (Sunnyvale, CA)
12 Ramya Edara MS Jan 2007 - Dec 2007 Member of Technical Staff, Arista Networks (Menlo Park, California)
13 Sachin Gaikwad MS Sep 2007 - Dec 2008 Member of Technical Staff, cohesity (Bangalore, India)
14 Aditya Kashyap MS Sep 2003 - Dec 2004 Director, Fixed Income Financing, Barclays (New York, NY)
15 Harikesavan Pathangi Krishnan MS Jan 2003 - Dec 2003 Software Engineer, Engineering department for WAFS product line, Packeteer, Inc. (South Plainfield, NJ)
16 Devaki Kulkarni MS Sep 2003 - Dec 2004 Staff Engineer, Performance and Storage groups, VMware, Inc. (Palo Alto, CA)
17 Ravikant R Malpani MS Sep 2011 - Dec 2012 Software Engineer at Hedvig inc (Santa Clara, CA).
18 Pradeep Shetty MS Aug 2010 - May 2012 Staff Engineer, CASL at Nimble Storage (San Jose, CA)
19 Karthikeyan Srinaivasan MS Jan 2009 - Dec 2009 Software Engineer, Rubrik Inc (Palo Alto, CA)
20 Gopala Suryanarayana MS Sep 2006 - Dec 2007 Staff Engineer, Storage and Availability vSAN Group, VMware, Inc. (Palo Alto, CA)
21 Sagar Trehan MS Sep 2012 - Dec 2013 Member of Technical Staff, CASL Performance Group - Engineering, Nimble Storage Inc (San Jose, USA)
22 Mohammad Nayyer Zubair MS Jan 2004 - Dec 2004 Software Developer, Bloomberg (New York, NY)
23 Rick Spillane BS/MS Jan 2005 - Dec 2007 Stony Brook U. CS Ph.D. program (Stony Brook, NY)
24 Ryan Leif Walsh BS (Honors) Jan 2009 - May 2009 Engineer, Simulations, Two Sigma Investments (New York, NY)
25 Mohammad Nayyer Zubair BS May 2003 - Dec 2003 Software Developer, Bloomberg (New York, NY)
26 Abhishek Rai temp-PhD Sep 2003 - Aug 2005 Sr. Director of Engineering, Databricks Inc. (Palo Alto, CA)

Sponsors:

# Sponsor Amount Period Type Title (click for award abstract)
1 NSF CSR--PDOS $561,727 2006-2009 Lead-PI Support for Atomic Sequences of File System Operations
2 NSF Trusted Computing (TC) $400,000 2003-2006 Sole PI A Layered Approach to Securing Network File Systems