SGI - Socket Gateway Interface

Application is written on Erlang. General design principles are: fast, low memory and modularity.

SGI gives possibility simple and smart way to connect to any server by TCP and has other protocols working under TCP. It supports two protocols:

Base components

Requirements

Try Samples

Sample 1 - Add a content from other languages inside of your Site.

This sample shows you how you can add "Busines Logic" (big or old) to your site using PHP files.

It is based on the sample from n2o.

$ git clone git://github.com/astronin/sgi
$ cd sgi/samples
$ ./mad deps compile plan
$ ./mad repl

Run php as fcgi server:

$ sudo service php5-fpm start

Now you can try it out: http://localhost:8000

Sample 2 - Your whole common site after erlang server and WebSocket instead of Ajax.

This sample shows you how you can run your site (written in other PL) with support of WebSocket. Forget about Ajax and do your page much more faster.

You have the following advantages compared to Ajax even in common web page:

Setup:
$ git clone git://github.com/astronin/sgi
$ cd sgi/samples

Change app in rebar.config:

$ vim samples/apps/rebar.config

{sub_dirs, [ "review" ]}. -> {sub_dirs, [ "review2" ]}.

Change app in sys.config:

{n2o, [{app,review2}]}

Run FPM:

$ sudo service php5-fpm start

Run Server:

$ ./mad deps compile plan
$ ./mad repl

Url: http://localhost:8000/site.php

Sample 3 - Like Sample 2, but using Python.

This sample shows you how you can run your site (wrote in Python and support uwsgi protocol) with support of WebSocket. For this you need to use server uWSGI.

Setup:
$ git clone git://github.com/astronin/sgi
$ cd sgi/samples

Change app in rebar.config:

$ vim samples/apps/rebar.config

{sub_dirs, [ "review" ]}. -> {sub_dirs, [ "review3" ]}.

Change app in sys.config:

{n2o, [{app,review3}]},
{sgi, [{servers, [
    [{name, default}, {address, localhost}, {port, 3031}]
]}]}

Run uWSGI:

$ uwsgi --socket 127.0.0.1:3031 --wsgi-file <your path clone>sgi/samples/cgi-scripts/python/myapp.py

Run Server:

$ ./mad deps compile plan
$ ./mad repl

Url: http://localhost:8000/

Sample 4 - Part of socket connection

This sample shows you how you can use TCP Client of this app. Thanks to the smart balancer Client can connect to any number of servers in different methods: priority or blurred.

Setup:
$ git clone git://github.com/astronin/sgi
$ cd sgi/samples

Change app in rebar.config:

$ vim samples/apps/rebar.config

{sub_dirs, [ "review" ]}. -> {sub_dirs, [ "review4" ]}.

Change app in sys.config:

{n2o, [{app,review4}]}

Change following settings in sys.config. Sample will start 10 servers with 5 processes. Client will connect with 5 sockets on each server.

{servers, [
    [{name, default}, {address, localhost}, {port, 10000}, {timeout, 60000}, {weight, 10}, {start_connections, 4}, {max_connections, 5}, {max_fails, 5}, {failed_timeout, 60}],
    [{name, aaa1},    {address, localhost}, {port, 10001}, {timeout, 60000}, {weight, 9},  {start_connections, 4}, {max_connections, 5}, {max_fails, 5}, {failed_timeout, 60}],
    [{name, aaa2},    {address, localhost}, {port, 10002}, {timeout, 60000}, {weight, 8},  {start_connections, 4}, {max_connections, 5}, {max_fails, 5}, {failed_timeout, 60}],
    [{name, aaa3},    {address, localhost}, {port, 10003}, {timeout, 60000}, {weight, 7},  {start_connections, 4}, {max_connections, 5}, {max_fails, 5}, {failed_timeout, 60}],
    [{name, aaa4},    {address, localhost}, {port, 10004}, {timeout, 60000}, {weight, 6},  {start_connections, 4}, {max_connections, 5}, {max_fails, 5}, {failed_timeout, 60}],
    [{name, aaa5},    {address, localhost}, {port, 10005}, {timeout, 60000}, {weight, 5},  {start_connections, 4}, {max_connections, 5}, {max_fails, 5}, {failed_timeout, 60}],
    [{name, aaa6},    {address, localhost}, {port, 10006}, {timeout, 60000}, {weight, 4},  {start_connections, 4}, {max_connections, 5}, {max_fails, 5}, {failed_timeout, 60}],
    [{name, aaa7},    {address, localhost}, {port, 10007}, {timeout, 60000}, {weight, 3},  {start_connections, 4}, {max_connections, 5}, {max_fails, 5}, {failed_timeout, 60}],
    [{name, aaa8},    {address, localhost}, {port, 10008}, {timeout, 60000}, {weight, 2},  {start_connections, 4}, {max_connections, 5}, {max_fails, 5}, {failed_timeout, 60}],
    [{name, aaa9},    {address, localhost}, {port, 10009}, {timeout, 60000}, {weight, 1},  {start_connections, 4}, {max_connections, 5}, {max_fails, 5}, {failed_timeout, 60}]
]},
{balancing_method, priority}, %blurred or priority

Run Server:

$ ./mad deps compile plan
$ ./mad repl

Change in sys.config next {balancing_method, blurred},

$ ./mad repl

Sample will create the two files: server_distribution(priority).csv and server_distribution(blurred).csv which shows difference between two methods of connection balancing to a server. Open files in Excel(or other) and insert XY(Scater) graph:

priority Below are results with next weights:

Server port | Weight --- | --- 10000 | 1 10001 | 2 10002 | 3 10003 | 4 10004 | 10 10005 | 9 10006 | 8 10007 | 7 10008 | 6 10009 | 5 prioritypriority

1. Basic usage

Use FastCGI protocol with N2O.

Configuration

Add the following section to sys.config

{sgi, [
    {servers, [
        [{name, default}, {address, localhost}, {port, 9000}, {timeout, 60000}, {weight, 2}, {start_connections, 10}, {max_connections, 1000}, {max_fails, 5}, {failed_timeout, 60}], % failed_timeout in seconds
        [{name, aaa},    {address, localhost}, {port, 9001}, {timeout, 60000}, {weight, 5}, {start_connections, 2}, {max_connections, 4}, {max_fails, 5}, {failed_timeout, 60}]
    ]},
        % max_connections - run N processes with 1 connection on each process. Count cannot be bigger then children of fcgi processes
    {balancing_method, priority}, % priority | blurred, priority is default
    {fcgi_timeout, 600000}, % 10 minutes
    {multiplexed, unknown}, % can be "1" | "0" | unknown
    {vhosts, [
        [
            {server_name, "phphost.com"}, % set your server name(domain), for local tests add line "127.0.0.1 phphost.com" into "/etc/hosts" (in Linux), "C:\Windows\System32\drivers\etc\hosts"(in Windows)
            {aliase, "localhost"},
            {root, "/home/roman/dev/sgi/samples/fcgi-scripts"}, % set you FULL path to your codes
            {index, "index.php"}, % default index file
            {rewrite, [{"*", "index.php"}]}
        ],
        [
            {server_name, "yourhost2.com"}, % set your server name(domain)
            {aliase, "localhost"},
            {root, "/usr/local/www/yourhost2.com"}, % set you full path to your codes
            {index, "index.php"} %% default index file
        ]
    ]}
    ]}

Steps of request to FCGI

Add deps to rebar.config:

{sgi, ".*", {git, "git://github.com/astronin/sgi", {tag, "master"}}}

Add initialization of fcgi protocol:

sgi_fcgi:init_fcgi()

Add new event in your file (index.erl):

event(#http{url = _Url, method = _Method, body = _Body} = Http)

Send data to the FastCGI Server:

{Ret, Status, Headers} = sgi_n2o_fcgi_handler:send(Http),

Using js code for returning data to the browser:

wf:wire("http.back(&#39;"++wf:to_list(js_escape(Ret))++"&#39;, "++wf:to_list(Status)++", "++wf:to_list(jsone:encode(Headers))++")");

Terminate fcgi protocol and clear memory:

fcgi_end()

2. Advanced usage

Application consists of two parts: Protocol Part and Connection Part.

Protocol Part:

FastCGI

The implementation of protocol includes two modules: N2O handler and protocol module. Other handlers can be written for other frameworks or servers.

Using N2O FastCGI Handler

@See Basic usage...

Using FastCGI Protocol

Module runnable as a process.

Start new fcgi process and start connection with fcgi server

{ok, Pid} = sgi_fcgi:start(1,1), % (FCGI_RESPONDER, FCGI_KEEP_CONN)

Send message

Pid ! {overall, self(), FCGIParams, has_body(Http), Body},

Or if you need more flexibility you can use next 3 commands: Send Params to server, this is pair of {Key,Value}

sgi_fcgi:params(Pid, FCGIParams),

Send data to a server, using real Pid so without excessive copying

Pid ! {5, Body},

Send the marker that the request is ended

sgi_fcgi:end_req(Pid),

Stop fcgi process with the release of resources

sgi_fcgi:stop(Pid),

Receive message with next tags

{sgi_fcgi_return, Out, Err} % common respones
sgi_fcgi_return_end % end of the requet, we can retun the answer to a browser
{sgi_fcgi_return_error, Err} % some error
{sgi_fcgi_timeout, Pid} % self timeout

Connection Part

Arbiter

Arbiter decides which stream is free (available), keeps busy socket (process), and it does not allow the use of extra resources.

Multiplexer

Run multiplexer if your application supports multiplexing. Even if you don't know, will be better to run multiplexer also, it helps your application not to get failed.

{ok, _Pid} = sgi_sup:start_child(sgi_multiplexer, {?MODULE, request_pid})

You should set callback function where multiplexer will get your local PID from a storage(ETS) by request_id of your request: {?MODULE, request_pid} Set callback after start process sgi_multiplexer:set_callback({M, F}) or set callback to each request {send, Request, PoolPid, M, F}.

Using
{ok, PoolPid} = sgi_arbiter:alloc(),
PoolPid ! {send, Data, self()},

or send data with multiplexer

sgi_multiplexer ! {send, Data, PoolPid},
handle_info({socket_return, Data}, State)

or was some error with connecting or sending a message

handle_info({socket_error, Data}, State)
sgi_arbiter:free(PoolPid),
Or easy way if you want Just Send data and just receive message

For example you want send quick (test) message and you do not want write many codes

{ok, Bin} = sgi_pool:once_call(Request),