Along with my team in the Cooperative Computing Lab, I enjoy building novel software systems that are used by people around the world to make new discoveries by turning ordinary applications into scalable distributed workflows.
TaskVine is our newest distributed execution system that harnesses the distributed storage devices embedded in HPC clusters to achieve extraordinary I/O rates for data intensive workflows, optimizing both software delivery as well and internal data movement throughout workflows. TaskVine allows for multi-model workflows combining both traditional task execution and serverless function oriented computing. (more) | |
Work Queue is an application framework for creating and managing dynamic manager-worker style programs that scale up to tens of thousands of machines on clusters, clouds, and grids. Work Queue has many advanced features for resource management, reliability, data management, and schedule. Applications are easy to write using Python libraries. Work Queue is used around the world to design dynamic scientific applications. (more) | |
Makeflow is a workflow system for parallel and distributed computing. Create massively parallel programs by joining together existing programs into large graphs. Use the classic Make-like syntax to get started, or use the JX syntax to programmatically generate large graphs. Execute workflows on your laptop or on large cluster using HTCondor, UGE, SLURM, and other systems. (more) | |
Basekernel is an open-source operating system kernel designed for teaching and research. It provides the basic abstractions of processes, scheduling, filesystems, and memory management, leaving room for the student or researcher to experiment with new methods. What makes Basekernel particularly novel is its "descending capability" model of permissions management. (more) |