Skip to search.

Breaking News Visit Yahoo! News for the latest.

×Close this window

json · JSON JavaScript Object Notation

The Yahoo! Groups Product Blog

Check it out!

Group Information

  • Members: 590
  • Category: Data Formats
  • Founded: Jul 19, 2005
  • Language: English
? Already a member? Sign in to Yahoo!

Yahoo! Groups Tips

Did you know...
Message search is now enhanced, find messages faster. Take it for a spin.

Messages

Advanced
Messages Help
Messages 1668 - 1697 of 1954   Oldest  |  < Older  |  Newer >  |  Newest
Messages: Show Message Summaries Sort by Date ^  
#1668 From: "mehdigholam@..." <mgholam@...>
Date: Thu Jun 9, 2011 6:12 am
Subject: TimeSpan type
mehdigholam...
Send Email Send Email
 
Hello,

As there is no JSON standard for serializing the .net TimeSpan type, are there
any suggestions?

We could do the following:
1. TimeSpan -> long value representing the ticks
2. TimeSpan -> string value like "ddd.HH:mm:ss.nnn"

#1669 From: Mark Slater <mark.slater@...>
Date: Thu Jun 9, 2011 7:32 am
Subject: Re: TimeSpan type
markosslater
Send Email Send Email
 
You might want to take a look at ISO 8061:
http://en.m.wikipedia.org/wiki/ISO_8601. I don't know about .Net, but many
languages have straightforward ways of converting to and from compliant strings.

Mark



On 9 Jun 2011, at 07:12, "mehdigholam@..." <mgholam@...> wrote:

> Hello,
>
> As there is no JSON standard for serializing the .net TimeSpan type, are there
any suggestions?
>
> We could do the following:
> 1. TimeSpan -> long value representing the ticks
> 2. TimeSpan -> string value like "ddd.HH:mm:ss.nnn"
>
>


[Non-text portions of this message have been removed]

#1670 From: John Cowan <cowan@...>
Date: Thu Jun 9, 2011 1:53 pm
Subject: Re: TimeSpan type
johnwcowan
Send Email Send Email
 
mehdigholam@... scripsit:

> 2. TimeSpan -> string value like "ddd.HH:mm:ss.nnn"

The ISO 8601 standard represents a duration in days, hours, minutes,
and seconds thus:  "PnnnDTnnHnnMnn.nnnS".  "P" indicates a duration and
is mandatory.  The nnnD, nnH, nnM, and nn.nnnS parts can be omitted if
they are zero, and you do not need to zero-pad the numeric values.  "T"
separates the date part from the time part, and is mandatory unless all
time values are omitted.

So P1D is one day, PT15H2S is fifteen hours and two seconds,
P1DT2H5M6.55S is one day, two hours, six minutes, and 6.550 seconds.
ISO 8601 has lots more bells and whistles, but this suffices for TimeSpan.

--
What asininity could I have uttered     John Cowan <cowan@...>
that they applaud me thus?              http://www.ccil.org/~cowan
         --Phocion, Greek orator

#1671 From: "Paul C. Bryan" <paul.bryan@...>
Date: Thu Jun 9, 2011 6:08 pm
Subject: JSON Patch Internet-Draft 01
paul.bryan@...
Send Email Send Email
 
The second draft of the JSON Patch proposal has been published:
http://tools.ietf.org/html/draft-pbryan-json-patch-01

Feedback would be most appreciated. The mailing list for this proposal
is:
https://groups.google.com/group/json-patch

Paul


[Non-text portions of this message have been removed]

#1672 From: Petri Lehtinen <petri@...>
Date: Sat Jun 11, 2011 8:19 pm
Subject: Jansson 2.1 released
akhern...
Send Email Send Email
 
Jansson 2.1 is out. This release adds a few new features and fixes
some minor bugs.

A new decoding function, json_loadb(), was added for decoding buffers
with length. The most important thing is that the input buffer need
not be null terminated.

Two new decoding flags and one new encoding flag were introduced:

* JSON_REJECT_DUPLICATES: Issue a decoding error if any JSON object in
   the input contins duplicate keys.

* JSON_DISABLE_EOF_CHECK: Stop decoding after a valid JSON input. This
   allows other data after the JSON data.

* JSON_ENCODE_ANY: Allow encoding any JSON value. Without this flag,
   only arrays and objects can be encoded as the root value.

Bugfixes:

* Fix a memory leak when memory allocation fails in json_object_set()
   and friends.

* Clear errno before calling strtod() for better portability (MINGW in
   this case).

* Avoid set-but-not-used warning/error when building with the newest
   GCC.

Download source: http://www.digip.org/jansson/releases/jansson-2.1.tar.gz
View documentation: http://www.digip.org/jansson/doc/2.1/
Changelog: http://www.digip.org/jansson/doc/2.1/changes.html#version-2-1


What is Jansson?
----------------

Jansson is a C library for encoding, decoding and manipulating JSON data.
It features:

* Simple and intuitive API and data model
* Comprehensive documentation
* No dependencies on other libraries
* Full Unicode support (UTF-8)
* Extensive test suite

Jansson is licensed under the MIT license.

For more details, see http://www.digip.org/jansson/.


Petri Lehtinen

#1673 From: "lambdacreations" <borislav.iordanov@...>
Date: Tue Jun 14, 2011 6:49 pm
Subject: and...another Java JSON
lambdacreations
Send Email Send Email
 
Hi all,

After reading up and using most the major Java JSON libraries, I thought I could
do something better (at least for needs similar
to my own's) and here is the result:

http://sharegov.blogspot.com/2011/06/json-library.html

The idea was to do an API that's simple, minimal and elegant, focusing
exclusively on JSON as a data structure.

Hoping that others will find it useful and pleasant to work with.

Best,
Boris

#1674 From: jonathan wallace <ninja9578@...>
Date: Fri Jul 15, 2011 2:06 pm
Subject: New libjson release
ninja9578
Send Email Send Email
 
I got a ton of requests for some security features, so I added DoS attack
detection and prevention, also made it faster and fixed some bugs.

https://sourceforge.net/projects/libjson/


[Non-text portions of this message have been removed]

#1675 From: keenerd <keenerd@...>
Date: Fri Aug 12, 2011 9:40 pm
Subject: Json tool for the shell
keenerd@...
Send Email Send Email
 
I noticed there was a small gap on json.org - there are no libraries
intended for shell script processing.  So I wrote one.

http://kmkeen.com/jshon/

Jshon provides a lightweight and simple means to access, modify,
create and introspect json structures from a *nix shell.  My favorite
use so far was a guy who uses Jshon on his router (not enough space to
install a big langauge) and talks to his electronic thermostat with
it.  Jshon is still relatively young, the only distro using and
packaging it is Arch Linux.

While based on Jansson, I feel it counts as an original work because
it presents an entirely new way of editing and manipulating json.  A
traditional library converts json into native structures (and back)
but Bash/Zsh/etc can not support such complex data structures.  So
Jshon uses a novel stack-based interface that is very simple for
common cases.

It is my hope that Jshon is considered sufficiently useful to be
included on json.org.

-Kyle Keen
http://kmkeen.com

#1676 From: Stephan Beal <sgbeal@...>
Date: Fri Aug 12, 2011 10:00 pm
Subject: Re: Json tool for the shell
stephan.beal
Send Email Send Email
 
On Fri, Aug 12, 2011 at 11:40 PM, keenerd <keenerd@...> wrote:

> **
>
>  noticed there was a small gap on json.org - there are no libraries
> intended for shell script processing. So I wrote one.
>
> http://kmkeen.com/jshon/
>
> Jshon provides a lightweight and simple means to access, modify,
> create and introspect json structures from a *nix shell.
>

Somewhat related, but not the same thing:

http://fossil.wanderinghorse.net/repos/cson/index.cgi/wiki?name=select-to-json

it's for running queries from sqlite3 or mysql dbs and getting the results
as json to stdout.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/


[Non-text portions of this message have been removed]

#1677 From: Stephan Beal <sgbeal@...>
Date: Fri Aug 12, 2011 10:02 pm
Subject: Re: Json tool for the shell
stephan.beal
Send Email Send Email
 
On Sat, Aug 13, 2011 at 12:00 AM, Stephan Beal <sgbeal@...>wrote:

> http://kmkeen.com/jshon/
>>
>>
http://fossil.wanderinghorse.net/repos/cson/index.cgi/wiki?name=select-to-json
>>
>
After a closer look at the first one, these two could be used together -
select-to-json feeding jshon.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/


[Non-text portions of this message have been removed]

#1678 From: Petri Lehtinen <petri@...>
Date: Sat Sep 3, 2011 10:28 am
Subject: Jansson 2.2 released
akhern...
Send Email Send Email
 
Jansson 2.2 is out. This release adds a new encoding function and
fixes some minor bugs and documentation glitches.

The new encoding function, json_dump_callback(), can be used to send
encoder's output to a callback function.

Download source: http://www.digip.org/jansson/releases/jansson-2.2.tar.gz
View documentation: http://www.digip.org/jansson/doc/2.2/
Changelog: http://www.digip.org/jansson/doc/2.2/changes.html#version-2-2


What is Jansson?
----------------

Jansson is a C library for encoding, decoding and manipulating JSON data.
It features:

* Simple and intuitive API and data model
* Comprehensive documentation
* No dependencies on other libraries
* Full Unicode support (UTF-8)
* Extensive test suite

Jansson is licensed under the MIT license.

For more details, see http://www.digip.org/jansson/.


Petri Lehtinen

#1679 From: "pozzugno" <pozzugno@...>
Date: Tue Sep 6, 2011 4:30 pm
Subject: Is JSON suitable for embedded applications?
pozzugno
Send Email Send Email
 
Dear group,

I'm a firmware developer on small embedded platform.

Until now, I used to read/write the in-memory configuration C structures (with
padding bytes) from/to a non-volatile memory.
This approach was sufficient for my old needs.

Actually I want to improve the configuration file format, most of all to face
the upgrade situations. When I create a new version of my application, usually I
have a new configuration C structure, so a new file format for the
configuration. After the upgrade, the software should be capable to read the
configuration in the old format, make the relevant changes and save the
configuration in the new format.

Could JSON data format be useful for me? Consider that my C compiler doesn't
support malloc()/free() functionalities, but only static and automatic variable
allocations. Is there a JSON C implementation suitable for small embedded
applications, without malloc()/free() and capable reading/writing without
storing the entire configuration in RAM?

Any suggestion for other data format?

#1680 From: jonathan wallace <ninja9578@...>
Date: Tue Sep 6, 2011 5:12 pm
Subject: Re: Is JSON suitable for embedded applications?
ninja9578
Send Email Send Email
 
I don't see why not.  JSON is lightweight, I use it for configuration all the
time.  I maintain a project called libjson that might work well for you.
 There are ways to configure it so that it never calls malloc/free, and it has
compiling options for embedded systems.

If you want to know how to do this with libjson, you can email me directly, or
ask on the sourceforge page.
 

________________________________
From: pozzugno <pozzugno@...>
To: json@yahoogroups.com
Sent: Tuesday, September 6, 2011 12:30 PM
Subject: [json] Is JSON suitable for embedded applications?


 
Dear group,

I'm a firmware developer on small embedded platform.

Until now, I used to read/write the in-memory configuration C structures (with
padding bytes) from/to a non-volatile memory.
This approach was sufficient for my old needs.

Actually I want to improve the configuration file format, most of all to face
the upgrade situations. When I create a new version of my application, usually I
have a new configuration C structure, so a new file format for the
configuration. After the upgrade, the software should be capable to read the
configuration in the old format, make the relevant changes and save the
configuration in the new format.

Could JSON data format be useful for me? Consider that my C compiler doesn't
support malloc()/free() functionalities, but only static and automatic variable
allocations. Is there a JSON C implementation suitable for small embedded
applications, without malloc()/free() and capable reading/writing without
storing the entire configuration in RAM?

Any suggestion for other data format?




[Non-text portions of this message have been removed]

#1681 From: "Douglas Crockford" <douglas@...>
Date: Tue Sep 6, 2011 5:28 pm
Subject: Silicon Valley Codecamp, October 8th and 9th, 2010
douglascrock...
Send Email Send Email
 
Silicon Valley Codecamp is a free software development conference held at
Foothill College in Los Altos, California. I will be presenting there again this
year. I hope to see you there.

Register at http://www.siliconvalley-codecamp.com/

#1682 From: "Douglas Crockford" <douglas@...>
Date: Tue Sep 6, 2011 5:36 pm
Subject: Re: Silicon Valley Codecamp, October 8th and 9th, 2010
douglascrock...
Send Email Send Email
 
--- In json@yahoogroups.com, "Douglas Crockford" <douglas@...> wrote:

> Silicon Valley Codecamp is a free software development conference held at
Foothill College in Los Altos, California. I will be presenting there again this
year. I hope to see you there.

> Register at http://www.siliconvalley-codecamp.com/

Update: This year only, it will be held in 2011.

#1683 From: "pozzugno" <pozzugno@...>
Date: Tue Sep 6, 2011 6:43 pm
Subject: Re: Is JSON suitable for embedded applications?
pozzugno
Send Email Send Email
 
--- In json@yahoogroups.com, jonathan wallace <ninja9578@...> wrote:
>
> I don't see why not.  JSON is lightweight, I use it for configuration all the
time.  I maintain a project called libjson that might work well for you.
 There are ways to configure it so that it never calls malloc/free, and it has
compiling options for embedded systems.
>
> If you want to know how to do this with libjson, you can email me directly, or
ask on the sourceforge page.

It sounds interesting, I'll look at it in short time. Thank you for your
suggestion.

#1684 From: "Douglas Crockford" <douglas@...>
Date: Fri Sep 9, 2011 1:36 pm
Subject: JSON-DA
douglascrock...
Send Email Send Email
 
The JSON page is now available in Danish, thanks to Michael Kramer.

http://json.org/json-da.html

#1685 From: "rkalla123" <rkalla@...>
Date: Wed Sep 21, 2011 1:15 pm
Subject: Universal Binary JSON Specification
rkalla123
Send Email Send Email
 
Hey Guys,

I am currently working on what I hope to be a 1:1 binary JSON specification (no
custom data types supported like BSON or BJSON, just binary representations of
the core JSON spec) and would appreciate a few extra eyes on it if anyone had
interest in reading through the 2nd draft:

https://docs.google.com/document/d/12SimAfBVcl8Fd-lr_SSBkM5B_PyEhDRfhgu1Lzvfpfw/\
edit?hl=en_US

The specification is fundamentally very simple, essentially breaking down to a
single construct that is used throughout:

[1-byte ASCII marker indicating type][4-byte int32 size][binary data]

My goals for the spec have been to strike what needs to be the perfect balance
between simplicity and verbosity in the binary data for the purposes of fast
parsing of the binary data. I also kept a close eye on defining
streaming-friendly constructs (no scanning for null-terminators) as well as
removing duplication of information when unnecessary.

The spec current defines 7 data types and the 2 collection types in JSON:
null
true
false
byte
int32
int64
double
string
object
array

The only difference from JSON being that "Number" is broken out into:
int32, int64 and double types for the purposes of making parsing of the values
as efficient as possible in Java, C, C#, Python, Erlang, PHP and any other
language that has multiple concepts of the different types of numbers it can
represent.

Using the test-data from the popular JVM-Serializer Benchmark project
(https://github.com/eishay/jvm-serializers/wiki) in Java, I show a 28% reduction
in file size along with a 73% faster deserialization step as compared to
Jackson's ObjectMapper.

NOTE: Jackson is currently the fastest JSON parsing library in Java, however I
believe there are ways to make the ObjectMapper run faster i just haven't tuned
it yet so 73% is likely not representative of final numbers.

My goals are to gun for (on average) a 30% reduction in file size and 50% faster
serialization/deserialization processing.

For what it is worth, I *expect* serialization/deserialization to run on par
with Java manual encoding/decoding which is labeled "java-M" in the JVM
Serialization Benchmark results (3rd fastest).

Any feedback, corrections or suggestions are all appreciated. Once I get some
more refinements into the spec and eyes on it to ensure there are no glaring
omissions everything will be published to http://ubjson.org and cultivated for
the betterment of programmers everywhere. There is nothing at the domain yet.

My wish is to contribute this back to the developer community leveraging JSON as
a companion spec if the community deems it beneficial.

Best,
Riyad

#1686 From: Stephan Beal <sgbeal@...>
Date: Wed Sep 21, 2011 1:23 pm
Subject: Re: Universal Binary JSON Specification
stephan.beal
Send Email Send Email
 
On Wed, Sep 21, 2011 at 3:15 PM, rkalla123 <rkalla@...> wrote:

> **
>
> The only difference from JSON being that "Number" is broken out into:
> int32, int64 and double types for the purposes of making parsing of the
>

Keep in mind that JSON does not specify any required numeric precision, and
some platforms cannot use int64. (JavaScript specifies 53 bits of precision,
in case it matters.) e.g. in C89 there is no _portable_ int64 construct
(that was introduced with C99, but lots of projects still use/require C89
because of the very different levels of C99 support in various compilers). i
know that Java is everyone's special baby, but some of us actually write
JSON-consuming/producing C89 code. In the world of C++, Google's v8
JavaScript engine doesn't support 64-bit integers: numbers >32 bits need to
be doubles on that platform.

In any case, i currently have a use case which will eventually require some
type of binary support, and i will be reading through what you've posted.
Thanks for sharing :).

Happy Hacking!


--
----- stephan beal
http://wanderinghorse.net/home/stephan/


[Non-text portions of this message have been removed]

#1687 From: "rkalla123" <rkalla@...>
Date: Wed Sep 21, 2011 1:26 pm
Subject: Re: Universal Binary JSON Specification
rkalla123
Send Email Send Email
 
Addendum:

"Number" is broken into 4 types... it is early here:
byte, int32, int64, double

#1688 From: "rkalla123" <rkalla@...>
Date: Wed Sep 21, 2011 1:58 pm
Subject: Re: Universal Binary JSON Specification
rkalla123
Send Email Send Email
 
Stephan,

Great feedback, these are exactly the kind of nuances I wanted to uncover and
discuss.

When I dug through the language specs looking for the most well-supported
numeric types before settling on the given 4, it seemed of all the ones I
checked JavaScript was the only one that didn't have native int64 support as you
mentioned.

The Chrome team suggests treating it like two int32s:
http://code.google.com/p/v8/issues/detail?id=1339

I found no way to deserialize JavaScript objects directly from a server-provided
byte stream which I anticipate meaning that this binary format doesn't benefit
users in the server-to-Browser communication workflow (which is already highly
optimized for text-based JSON) but as a general application binary interchange
format. Like values stored as files on the server or processes communicating
with one another.

It was my thinking that this leaves the int64 inclusion in the binary spec in a
relatively safe position as the primary use case will be between languages that
support 64-bit ints.

Said another way, I certainly see your point, but I am trying to avoid a "cut
off nose despite your face" situation with forward-thinking enhancements to
JavaScript engines in the next few years that I assume will eventually give us
64-bit integers as the JS spec advances.

I would hate then to be prompted to add 64-bit integers back to the binary spec
after it has been out and in circulation for a few years.

ASIDE: If I am missing a native way to reconstitute JavaScript objects from a
server-provided byte stream and using this binary format for server-browser
communication is an optimized reality, please correct me. I was unable to dig up
methods to do this.

--- In json@yahoogroups.com, Stephan Beal <sgbeal@...> wrote:
>
> On Wed, Sep 21, 2011 at 3:15 PM, rkalla123 <rkalla@...> wrote:
>
> > **
> >
> > The only difference from JSON being that "Number" is broken out into:
> > int32, int64 and double types for the purposes of making parsing of the
> >
>
> Keep in mind that JSON does not specify any required numeric precision, and
> some platforms cannot use int64. (JavaScript specifies 53 bits of precision,
> in case it matters.) e.g. in C89 there is no _portable_ int64 construct
> (that was introduced with C99, but lots of projects still use/require C89
> because of the very different levels of C99 support in various compilers). i
> know that Java is everyone's special baby, but some of us actually write
> JSON-consuming/producing C89 code. In the world of C++, Google's v8
> JavaScript engine doesn't support 64-bit integers: numbers >32 bits need to
> be doubles on that platform.
>
> In any case, i currently have a use case which will eventually require some
> type of binary support, and i will be reading through what you've posted.
> Thanks for sharing :).
>
> Happy Hacking!
>
>
> --
> ----- stephan beal
> http://wanderinghorse.net/home/stephan/
>
>
> [Non-text portions of this message have been removed]
>

#1689 From: Stephan Beal <sgbeal@...>
Date: Wed Sep 21, 2011 2:12 pm
Subject: Re: Re: Universal Binary JSON Specification
stephan.beal
Send Email Send Email
 
On Wed, Sep 21, 2011 at 3:58 PM, rkalla123 <rkalla@...> wrote:

> **
>
> The Chrome team suggests treating it like two int32s:
> http://code.google.com/p/v8/issues/detail?id=1339
>

i've been using doubles for that purpose in that particular JS engine (v8).

I found no way to deserialize JavaScript objects directly from a
> server-provided byte stream which I anticipate meaning that this binary
> format doesn't benefit users in the server-to-Browser communication workflow
> (which
>
The "new" JS version (don't remember the version number) fills that gap. It
allows us to efficiently consume and produce binary data in JS. Trying to
use a JS Array to store binary data (e.g. as 1-4 bytes/element) has a HUGE
overhead because of the internal impl details of the Array class, which is
normally a linked list).


> It was my thinking that this leaves the int64 inclusion in the binary spec
> in a relatively safe position as the primary use case will be between
> languages that support 64-bit ints.
>

Sure, just as long as it's clear that this must be a "should" part of the
spec instead of a "must" because not all platforms can guaranty 64-bit int
handling.

Said another way, I certainly see your point, but I am trying to avoid a
> "cut off nose despite your face" situation with forward-thinking
> enhancements to JavaScript engines in the next few years that I assume will
> eventually give us 64-bit integers as the JS spec advances.
>
i agree, but just be aware that JSON itself does NOT specify ANY precision
requirements, probably because it cannot safely due so without also
inherently excluding a range of environments. That said... since you're
talking about encoding them, the limits of JSON do not apply to what you
encode, as long as the encoded data is JSON-compliant (which it obviously
is).

I would hate then to be prompted to add 64-bit integers back to the binary
> spec after it has been out and in circulation for a few years.
>
i sympathize entirely with that.

--
----- stephan beal
http://wanderinghorse.net/home/stephan/


[Non-text portions of this message have been removed]

#1690 From: "rkalla123" <rkalla@...>
Date: Wed Sep 21, 2011 3:29 pm
Subject: Re: Universal Binary JSON Specification
rkalla123
Send Email Send Email
 
--- In json@yahoogroups.com, Stephan Beal <sgbeal@...> wrote:
> The "new" JS version (don't remember the version number) fills that gap. It
> allows us to efficiently consume and produce binary data in JS. Trying to
> use a JS Array to store binary data (e.g. as 1-4 bytes/element) has a HUGE
> overhead because of the internal impl details of the Array class, which is
> normally a linked list).

Do you happen to have a link? This is something I'd like to know more about/play
with a bit.

Best,
Riyad

#1691 From: Stephan Beal <sgbeal@...>
Date: Wed Sep 21, 2011 3:32 pm
Subject: Re: Re: Universal Binary JSON Specification
stephan.beal
Send Email Send Email
 
On Wed, Sep 21, 2011 at 5:29 PM, rkalla123 <rkalla@...> wrote:

> **
> Do you happen to have a link? This is something I'd like to know more
> about/play with a bit.
>

Here's a couple...

http://www.lbi.co.uk/blog/latest/typed-arrays-in-javascript/
http://www.khronos.org/registry/typedarray/specs/latest/

--
----- stephan beal
http://wanderinghorse.net/home/stephan/


[Non-text portions of this message have been removed]

#1692 From: John Cowan <cowan@...>
Date: Wed Sep 21, 2011 3:39 pm
Subject: Re: Re: Universal Binary JSON Specification
johnwcowan
Send Email Send Email
 
Stephan Beal scripsit:

> http://www.lbi.co.uk/blog/latest/typed-arrays-in-javascript/
> http://www.khronos.org/registry/typedarray/specs/latest/

It's notable that neither of these provides an int64 type, only
signed and unsigned 8-bit, 16-bit, and 32-bit integers plus
short and long floats.

--
Dream projects long deferred             John Cowan <cowan@...>
usually bite the wax tadpole.            http://www.ccil.org/~cowan
         --James Lileks

#1693 From: Tatu Saloranta <tsaloranta@...>
Date: Wed Sep 21, 2011 4:45 pm
Subject: Re: Universal Binary JSON Specification
cowtowncoder
Send Email Send Email
 
On Wed, Sep 21, 2011 at 6:15 AM, rkalla123 <rkalla@...> wrote:
> Hey Guys,
>
> I am currently working on what I hope to be a 1:1 binary JSON specification
(no custom data types supported like BSON or BJSON, just binary representations
of the core JSON spec) and would appreciate a few extra eyes on it if anyone had
interest in reading through the 2nd draft:

You might be interested in an existing such specification called
Smile: http://wiki.fasterxml.com/SmileFormatSpec
which was specified about a year ago, has Java and C implementations,
and used by a few projects/products like ElasticSearch.

And for performance, Smile has been included on jvm-serializer
benchmark for a while now, using Jackson data binding.

-+ Tatu +-

#1694 From: Tatu Saloranta <tsaloranta@...>
Date: Wed Sep 21, 2011 4:48 pm
Subject: Re: Universal Binary JSON Specification
cowtowncoder
Send Email Send Email
 
Btw, forgot to add:

On Wed, Sep 21, 2011 at 6:15 AM, rkalla123 <rkalla@...> wrote:
...
> Using the test-data from the popular JVM-Serializer Benchmark project
(https://github.com/eishay/jvm-serializers/wiki) in Java, I show a 28% reduction
in file size along with a 73% faster deserialization step as compared to
Jackson's ObjectMapper.

It would be great if you could contribute this to jvm-serialziers, I
can help in getting it merged.

> NOTE: Jackson is currently the fastest JSON parsing library in Java, however I
believe there are ways to make the ObjectMapper run faster i just haven't tuned
it yet so 73% is likely not representative of final numbers.
>
> My goals are to gun for (on average) a 30% reduction in file size and 50%
faster serialization/deserialization processing.
>
> For what it is worth, I *expect* serialization/deserialization to run on par
with Java manual encoding/decoding which is labeled "java-M" in the JVM
Serialization Benchmark results (3rd fastest).

I actually suspect you are wrong there -- since you require
length-prefixing, writing is going to be slower, at least for longer
strings. :-)

But would be happy to be proven wrong.

-+ Tatu +-

#1695 From: Martin Cooper <mfncooper@...>
Date: Wed Sep 21, 2011 5:12 pm
Subject: Re: Universal Binary JSON Specification
mfncooper
Send Email Send Email
 
On Wed, Sep 21, 2011 at 6:15 AM, rkalla123 <rkalla@...> wrote:
> Hey Guys,
>
> I am currently working on what I hope to be a 1:1 binary JSON specification
(no custom data types supported like BSON or BJSON, just binary representations
of the core JSON spec) and would appreciate a few extra eyes on it if anyone had
interest in reading through the 2nd draft:
>
>
https://docs.google.com/document/d/12SimAfBVcl8Fd-lr_SSBkM5B_PyEhDRfhgu1Lzvfpfw/\
edit?hl=en_US
>

A couple of things from a quick skim of the document:

* "As with JSON, all Strings are encoding in UTF-8"
The *default* JSON encoding is UTF-8, but JSON itself may be
represented using UTF-8, UTF-16, or UTF-32.

* "Arrays are a flat list of same-typed values"
There is no requirement in JSON that the values be same-typed.

--
Martin Cooper


> The specification is fundamentally very simple, essentially breaking down to a
single construct that is used throughout:
>
> [1-byte ASCII marker indicating type][4-byte int32 size][binary data]
>
> My goals for the spec have been to strike what needs to be the perfect balance
between simplicity and verbosity in the binary data for the purposes of fast
parsing of the binary data. I also kept a close eye on defining
streaming-friendly constructs (no scanning for null-terminators) as well as
removing duplication of information when unnecessary.
>
> The spec current defines 7 data types and the 2 collection types in JSON:
> null
> true
> false
> byte
> int32
> int64
> double
> string
> object
> array
>
> The only difference from JSON being that "Number" is broken out into:
> int32, int64 and double types for the purposes of making parsing of the values
as efficient as possible in Java, C, C#, Python, Erlang, PHP and any other
language that has multiple concepts of the different types of numbers it can
represent.
>
> Using the test-data from the popular JVM-Serializer Benchmark project
(https://github.com/eishay/jvm-serializers/wiki) in Java, I show a 28% reduction
in file size along with a 73% faster deserialization step as compared to
Jackson's ObjectMapper.
>
> NOTE: Jackson is currently the fastest JSON parsing library in Java, however I
believe there are ways to make the ObjectMapper run faster i just haven't tuned
it yet so 73% is likely not representative of final numbers.
>
> My goals are to gun for (on average) a 30% reduction in file size and 50%
faster serialization/deserialization processing.
>
> For what it is worth, I *expect* serialization/deserialization to run on par
with Java manual encoding/decoding which is labeled "java-M" in the JVM
Serialization Benchmark results (3rd fastest).
>
> Any feedback, corrections or suggestions are all appreciated. Once I get some
more refinements into the spec and eyes on it to ensure there are no glaring
omissions everything will be published to http://ubjson.org and cultivated for
the betterment of programmers everywhere. There is nothing at the domain yet.
>
> My wish is to contribute this back to the developer community leveraging JSON
as a companion spec if the community deems it beneficial.
>
> Best,
> Riyad
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>

#1696 From: John Cowan <cowan@...>
Date: Wed Sep 21, 2011 5:22 pm
Subject: Re: Universal Binary JSON Specification
johnwcowan
Send Email Send Email
 
Martin Cooper scripsit:
> On Wed, Sep 21, 2011 at 6:15 AM, rkalla123 <rkalla@...> wrote:
> > Hey Guys,
> >
> > I am currently working on what I hope to be a 1:1 binary JSON specification
(no custom data types supported like BSON or BJSON, just binary representations
of the core JSON spec) and would appreciate a few extra eyes on it if anyone had
interest in reading through the 2nd draft:
> >
> >
https://docs.google.com/document/d/12SimAfBVcl8Fd-lr_SSBkM5B_PyEhDRfhgu1Lzvfpfw/\
edit?hl=en_US
> >
>
> A couple of things from a quick skim of the document:
>
> * "As with JSON, all Strings are encoding in UTF-8"
> The *default* JSON encoding is UTF-8, but JSON itself may be
> represented using UTF-8, UTF-16, or UTF-32.
>
> * "Arrays are a flat list of same-typed values"
> There is no requirement in JSON that the values be same-typed.

Fortunately, these points don't affect the actual protocol,
just the explanation.

--
LEAR: Dost thou call me fool, boy?      John Cowan
FOOL: All thy other titles              http://www.ccil.org/~cowan
              thou hast given away:      cowan@...
       That thou wast born with.

#1697 From: Stephan Beal <sgbeal@...>
Date: Wed Sep 21, 2011 5:44 pm
Subject: Re: Universal Binary JSON Specification
stephan.beal
Send Email Send Email
 
On Wed, Sep 21, 2011 at 6:45 PM, Tatu Saloranta <tsaloranta@...>wrote:

> **
> You might be interested in an existing such specification called
> Smile: http://wiki.fasterxml.com/SmileFormatSpec
> which was specified about a year ago, has Java and C implementations,
> and used by a few projects/products like ElasticSearch.
>

Thank you for that. Smile's requirement that impls be capable of supporting
"shared strings" seems a bit draconian to me, though. That adds non-trivial
parser/writer infrastructure which would otherwise not be required
(especially in C, which doesn't have standard containers we can use to store
such strings/references in).

--
----- stephan beal
http://wanderinghorse.net/home/stephan/


[Non-text portions of this message have been removed]

Messages 1668 - 1697 of 1954   Oldest  |  < Older  |  Newer >  |  Newest
Add to My Yahoo!      XML What's This?

Copyright © 2010 Yahoo! Inc. All rights reserved.
Privacy Policy - Terms of Service - Guidelines NEW - Help