Welcome to Two Sigma's Cook Scheduler!
What is Cook?
Core concepts is a good place to start to learn more.
Check the changelog for release info.
In this repository, you'll find several subprojects, each of which has its own documentation.
scheduler- This is the actual Mesos framework, Cook. It comes with a JSON REST API.
jobclient- This includes the Java and Python APIs for Cook, both of which use the REST API under the hood.
spark- This contains the patch to Spark to enable Cook as a backend.
Please visit the
scheduler subproject first to get started.
GCP_PROJECT_NAME=<gcp_project_name> vagrant up --provider=virtualboxto create the dev environment
vagrant sshto ssh into the dev environment
gcloud auth loginto login to Google cloud
bin/make-gke-test-clustersto create GKE clusters
bin/start-datomic.shto start Datomic (Cook database)
lein exec -p datomic/data/seed_k8s_pools.clj $COOK_DATOMIC_URIto seed some Cook pools in the database
bin/run-local-kubernetes.shto start the Cook scheduler
To test a simple job submission:
cs submit --pool k8s-alpha --cpu 0.5 --mem 32 --docker-image gcr.io/google-containers/alpine-with-bash:1.0 lsto submit a simple job
cs show <job_uuid>to show the status of your job (it should eventually show Success)
To run automated tests:
lein test :all-but-benchmarkto run unit tests
cd ../integration && pytest -m 'not cli'to run integration tests
cd ../integration && pytest -k test_basic_submit -n 0 -sto run a particular integration test
bin/build-docker-image.shto build the Cook scheduler image
../travis/minimesos upto start Mesos and ZooKeeper using minimesos
bin/run-docker.shto start the Cook scheduler
In order to accept your code contributions, please fill out the appropriate Contributor License Agreement in the
cla folder and submit it to email@example.com.
Apache Mesos is a trademark of The Apache Software Foundation. The Apache Software Foundation is not affiliated, endorsed, connected, sponsored or otherwise associated in any way to Two Sigma, Cook, or this website in any manner.
© Two Sigma Open Source, LLC