gri

gridcontrol

Networked Process Manager to execute functions in a Computer Grid

Showing:

Popularity

Downloads/wk

9

GitHub Stars

580

Maintenance

Last Commit

5yrs ago

Contributors

5

Package

Dependencies

28

License

Apache-2.0

Type Definitions

Tree-Shakeable

No?

Categories

Readme

Build Status

GridControl

GridControl provisions and links multiple servers together to form a Grid.

Files are synchronized, Opinionated Pub/Sub system is implemented, Servers get linked together.

You develop, you play, in a scalable way. The more Servers you add to the Grid, the more calculation power you get.

Welcome to the Grid

5 minutes to get started. By the authors of PM2.

Behind the scenes: GridControl is a network layer built on top of PM2 allowing file synchronization, inter-process communication via an opionated PUB/SUB system and a wide-range system discovery

Features

  • 0 conf Auto discovery system via multiple DNS
  • 0 conf P2P application source sharing
  • Ecosystem Grid management toolbox (CLI, provisioning, Logs, Monitoring)
  • Secure Diffie Hellman key exchange and password authentication
  • Decentralized Each Node can trigger actions executed by another Nodes
  • Fast Grid interconnected via TCP sockets
  • Fast Services are started once, then stay alive waiting for inputs. This saves non-negligible startup time
  • Polyglot Services can be written in any language
  • Compatible with Amazon Lambda, Google Cloud Functions
  • Rock Solid PM2 behind the scene for process management and cluster capabilities
  • And a lot more like Buffering, Retry on Failure...

Creating a Grid

Install your Swiss Army Knife to manage a Grid:

$ npm install grid-cli -g

Now the bin grid is available via the CLI.

Commands

1/ Generate a new Gridfile in the current path that contains grid name, grid password, host and SSH keys:

$ grid new

The Gridfile will look like this:

grid_name     = 'grid-name'
grid_password = 'xxxx'

servers = [
  'user@ip1',
  'user@ip2',
  'user@ip3'
]

ssh_key = '''
1024_PRIVATE_SSH_KEY
'''

ssh_public_key = '''
PUBLIC_SSH_KEY
'''

Change each attribute to the desired value. Note that an SSH client should be running on the defaut port (22) on each remote machine

2/ Provision every host listed in the Gridfile:

$ grid provision

This will copy the SSH pub key and install NVM, Node.js, PM2 and Gridcontrol This installation does not need ROOT access rights at any time

3/ Grid management

$ grid dash

Grid dashboard

Commands to manage your grid:

# List all nodes linked to the grid
$ grid ls

# Display Dashboard
$ grid dash

# Execute a command on each server
$ grid multissh <bash_command>

# Restart/Recover the current Grid
$ grid restart

# Upgrade Gridcontrol to latest version
$ grid upgrade

# Display realtime logs of all tasks
$ grid logs

# Monitor the whole Grid with Keymetrics
$ grid monitor <secret> <public>
$ grid unmonitor

# Interactively SSH into desired machine
$ grid ssh

Interact with the Grid

Now let's play with the Grid. You can generate a sample project by typing:

$ grid sample [project-name]
$ cd [project-name]
$ npm install

Now you'll have a project that looks like this:

.
├── index.js
├── package.json
└── tasks
    └── get-ip.js

Let's look at the content of tasks/get-ip.js, this is a task that will be propagated in the grid:

var request = require('request');

module.exports = function(context, cb) {
  request('https://api.ipify.org/?format=json', function (error, response, body) {
    if (error)
      return cb(error);

    if (!error && response.statusCode == 200) {
      return cb(null, body);
    }
  });
};

To call this function, look at how it's done in index.js:

var grid = require('grid-api').init({
  instances   : 1,
  env         : {
    NODE_ENV  : 'development'
  }
});

function triggerTask() {
  // 'get-ip' is the filename
  grid.exec('get-ip', function(err, data, server) {
    if (err) {
      console.log(err);
      return false;
    }
      console.log('Got response from server pub_ip=(%s) priv_ip=(%s):',
                server.public_ip,
                server.private_ip);
    console.log(data);
  });
}

grid.on('ready', function() {
  console.log('Gridcontrol Ready');
  setInterval(triggerTask, 1000);
});

Start the main application:

$ node index.js

At the beginning, only the local gridcontrol will respond. Once the other peers are synchronized they will also process the queries:

Got response from server pub_ip=(88.123.12.21) priv_ip=(10.2.2.1):
$HTML
Got response from server pub_ip=(88.123.12.22) priv_ip=(10.2.2.8):
$HTML
Got response from server pub_ip=(88.125.120.20) priv_ip=(10.2.2.10):
$HTML
Got response from server pub_ip=(88.123.23.42) priv_ip=(10.2.2.12):
$HTML

Distributed processing, on-premise!

Contributing

If you find any issues please open an issue on Github

For Contributing please refer to docs/CONTRIBUTING.md

License

Apache V2 (see LICENSE.txt)

Rate & Review

Great Documentation0
Easy to Use0
Performant0
Highly Customizable0
Bleeding Edge0
Responsive Maintainers0
Poor Documentation0
Hard to Use0
Slow0
Buggy0
Abandoned0
Unwelcoming Community0
100
No reviews found
Be the first to rate

Alternatives

No alternatives found

Tutorials

No tutorials found
Add a tutorial