jsx (v1.4.5)

an erlang application for consuming, producing and manipulating json. inspired by yajl

jsx is built via rebar and continuous integration testing provided courtesy travis

current status: Build Status

jsx is released under the terms of the MIT license

copyright 2010-2013 alisdair sullivan

index

quickstart

to build the library and run tests

$ rebar compile

$ rebar eunit

or, to build using hipe

$ rebar -C hipe.cfg compile

$ rebar -C hipe.cfg eunit

to convert a utf8 binary containing a json string into an erlang term

1> jsx:decode(<<"{\"library\": \"jsx\", \"awesome\": true}">>).

[{<<"library">>,<<"jsx">>},{<<"awesome">>,true}]

2> jsx:decode(<<"[\"a\",\"list\",\"of\",\"words\"]">>).

[<<"a">>, <<"list">>, <<"of">>, <<"words">>]

to convert an erlang term into a utf8 binary containing a json string

1> jsx:encode([{<<"library">>,<<"jsx">>},{<<"awesome">>,true}]).

<<"{\"library\": \"jsx\", \"awesome\": true}">>

2> jsx:encode([<<"a">>, <<"list">>, <<"of">>, <<"words">>]).

<<"[\"a\",\"list\",\"of\",\"words\"]">>

to check if a binary or a term is valid json

1> jsx:is_json(<<"[\"this is json\"]">>).

true

2> jsx:is_json("[\"this is not\"]").

false

3> jsx:is_term([<<"this is a term">>]).

true

4> jsx:is_term([this, is, not]).

false

to minify some json

1> jsx:minify(<<"{

  \"a list\": [

    1,

    2,

    3

  ]

}">>).

<<"{\"a list\":[1,2,3]}">>

to prettify some json

1> jsx:prettify(<<"{\"a list\":[1,2,3]}">>).

<<"{

  \"a list\": [

    1,

    2,

    3

  ]

}">>

description

jsx is an erlang application for consuming, producing and manipulating json

json has a spec but common usage differs subtly. it's common usage jsx attempts to address, with guidance from the spec

all json produced and consumed by jsx should be utf8 encoded text or a reasonable approximation thereof. ascii works too, but anything beyond that i'm not going to make any promises. especially not latin1

the spec thinks json values must be wrapped in a json array or object but everyone else disagrees so jsx allows naked json values by default. if you're a curmudgeon who's offended by this deviation here is a wrapper for you:

%% usage: `real_json(jsx:decode(JSON))`

real_json(Result) when is_list(Result) -> Result;

real_json(Result) when is_tuple(Result, 2) -> Result;

real_json(_) -> erlang:error(badarg).

json <-> erlang mapping

jsonerlang
numberinteger() and float()
stringbinary() and atom()
true, false and nulltrue, false and null
array[] and [JSON]
object[{}] and [{binary() OR atom(), JSON}]

incomplete input

jsx handles incomplete json texts. if a partial json text is parsed, rather than returning a term from your callback handler, jsx returns {incomplete, F} where F is a function with an identical API to the anonymous fun returned from decoder/3, encoder/3 or parser/3. it retains the internal state of the parser at the point where input was exhausted. this allows you to parse as you stream json over a socket or file descriptor, or to parse large json texts without needing to keep them entirely in memory

however, it is important to recognize that jsx is greedy by default. jsx will consider the parsing complete if input is exhausted and the json text is not unambiguously incomplete. this is mostly relevant when parsing bare numbers like <<"1234">>. this could be a complete json integer or just the beginning of a json integer that is being parsed incrementally. jsx will treat it as a whole integer. calling jsx with the optionexplicit_end reverses this behavior and never considers parsing complete until the incomplete function is called with the argument end_stream

data types

json_term()

json_term() = [json_term()]

    | [{binary() | atom(), json_term()}]

    | true

    | false

    | null

    | integer()

    | float()

    | binary()

    | atom()

the erlang representation of json. binaries should be utf8 encoded, or close at least

json_text()

json_text() = binary()

a utf8 encoded binary containing a json string

event()

event() = start_object

    | end_object

    | start_array

    | end_array

    | {key, binary()}

    | {string, binary()}

    | {integer, integer()}

    | {float, float()}

    | {literal, true}

    | {literal, false}

    | {literal, null}

    | end_json

token()

token() = event()

    | binary()

    | {number, integer() | float()}

    | integer()

    | float()

    | true

    | false

    | null

the representation used during syntactic analysis. you can generate this yourself and feed it to jsx:parser/3 if you'd like to define your own representations

option()

option() = replaced_bad_utf8

    | escaped_forward_slashes

    | single_quoted_strings

    | unescaped_jsonp

    | comments

    | escaped_strings

    | dirty_strings

    | ignored_bad_escapes

    | relax

    | explicit_end

jsx functions all take a common set of options. not all flags have meaning in all contexts, but they are always valid options. functions may have additional options beyond these. see individual function documentation for details

exports

encoder/3, decoder/3 & parser/3

decoder(Module, Args, Opts) -> Fun((JSONText) -> any())

encoder(Module, Args, Opts) -> Fun((JSONTerm) -> any())

parser(Module, Args, Opts) -> Fun((Tokens) -> any())



  Module = atom()

  Args = any()

  Opts = [option()]

  JSONText = json_text()

  JSONTerm = json_term()

  Tokens = token() | [token()]

jsx is a json compiler with interleaved tokenizing, syntactic analysis and semantic analysis stages. included are two tokenizers; one that handles json texts (decoder/3) and one that handles erlang terms (encoder/3). there is also an entry point to the syntactic analysis stage for use with user-defined tokenizers (parser/3)

all three functions return an anonymous function that takes the appropriate type of input and returns the result of performing semantic analysis, the tuple {incomplete, F} where F is a new anonymous function (see incomplete input) or a badarg error exception if syntactic analysis fails

Module is the name of the callback module

Args is any term that will be passed to Module:init/1 prior to syntactic analysis to produce an initial state

Opts are detailed here

check out callback module documentation for details of the callback module interface

decode/1,2

decode(JSON) -> Term

decode(JSON, Opts) -> Term



  JSON = json_text()

  Term = json_term()

  Opts = [option() | labels | {labels, Label} | {post_decode, F}]

    Label = binary | atom | existing_atom | attempt_atom

    F = fun((any()) -> any())

decode parses a json text (a utf8 encoded binary) and produces an erlang term

the option labels controls how keys are converted from json to erlang terms. binary (the default behavior) does no conversion beyond normal escaping. atom converts keys to erlang atoms and results in a badarg error if the keys fall outside the range of erlang atoms. existing_atom is identical to atom except it will not add new atoms to the atom table and will result in a badarg error if the atom does not exist. attempt_atom will convert keys to atoms when they exist, and leave them as binary otherwise

{post_decode, F} is a user defined function of arity 1 that is called on each output value (objects, arrays, strings, numbers and literals). it may return any value to be substituted in the returned term. for example:

1> F = fun(V) when is_list(V) -> V; (V) -> false end.

2> jsx:decode(<<"{\"a list\": [true, \"a string\", 1]}">>, [{post_decode, F}]).

[{<<"a list">>, [false, false, false]}]

declaring more than one post-decoder will result in a badarg error exception

raises a badarg error exception if input is not valid json

encode/1,2

encode(Term) -> JSON

encode(Term, Opts) -> JSON



  Term = json_term()

  JSON = json_text()

  Opts = [option() | {pre_encode, F} | space | {space, N} | indent | {indent, N}]

    F = fun((any()) -> any())

    N = pos_integer()

encode converts an erlang term into json text (a utf8 encoded binary)

the option {space, N} inserts N spaces after every comma and colon in your json output. space is an alias for {space, 1}. the default is {space, 0}

the option {indent, N} inserts a newline and N spaces for each level of indentation in your json output. note that this overrides spaces inserted after a comma. indent is an alias for {indent, 1}. the default is {indent, 0}

{pre_encode, F} is a user defined function of arity 1 that is called on each input value. it may return any valid json value to be substituted in the returned json. for example:

1> F = fun(V) when is_list(V) -> V; (V) -> false end.

2> jsx:encode([{<<"a list">>, [true, <<"a string">>, 1]}], [{pre_encode, F}]).

<<"{\"a list\": [false, false, false]}">>

declaring more than one pre-encoder will result in a badarg error exception

raises a badarg error exception if input is not a valid erlang representation of json

format/1,2

format(JSON) -> JSON

format(JSON, Opts) -> JSON



  JSON = json_text()

  Opts = [option() | space | {space, N} | indent | {indent, N}]

    N = pos_integer()

format parses a json text (a utf8 encoded binary) and produces a new json text according to the format rules specified by Opts

the option {space, N} inserts N spaces after every comma and colon in your json output. space is an alias for {space, 1}. the default is {space, 0}

the option {indent, N} inserts a newline and N spaces for each level of indentation in your json output. note that this overrides spaces inserted after a comma. indent is an alias for {indent, 1}. the default is {indent, 0}

raises a badarg error exception if input is not valid json

minify/1

minify(JSON) -> JSON



  JSON = json_text()

minify parses a json text (a utf8 encoded binary) and produces a new json text stripped of whitespace

raises a badarg error exception if input is not valid json

prettify/1

prettify(JSON) -> JSON



  JSON = json_text()

prettify parses a json text (a utf8 encoded binary) and produces a new json text equivalent to format(JSON, [{space, 1}, {indent, 2}])

raises a badarg error exception if input is not valid json

is_json/1,2

is_json(MaybeJSON) -> true | false

is_json(MaybeJSON, Opts) -> true | false



  MaybeJSON = any()

  Opts = options()

returns true if input is a valid json text, false if not

what exactly constitutes valid json may be altered

is_term/1,2

is_term(MaybeJSON) -> true | false

is_term(MaybeJSON, Opts) -> true | false



  MaybeJSON = any()

  Opts = options()

returns true if input is a valid erlang representation of json, false if not

what exactly constitutes valid json may be altered via options

callback exports

the following functions should be exported from a jsx callback module

Module:init/1

Module:init(Args) -> InitialState



  Args = any()

  InitialState = any()

whenever any of encoder/3, decoder/3 or parser/3 are called, this function is called with the Args argument provided in the calling function to obtain InitialState

Module:handle_event/2

Module:handle_event(Event, State) -> NewState



  Event = [event()]

  State = any()

  NewState = any()

semantic analysis is performed by repeatedly calling handle_event/2 with a stream of events emitted by the tokenizer and the current state. the new state returned is used as the input to the next call to handle_event/2. the following events must be handled:

acknowledgements

jsx wouldn't be what it is without the contributions of paul davis, lloyd hilaiel, john engelhart, bob ippolito, fernando benavides, alex kropivny, steve strong, michael truog, devin torres, dmitry kolesnikov, emptytea, john daily, ola bäckström, joseph crowe, patrick gombert, eskuat and max lapshin