Hi Kris! I think this is a good group to have. :)

One quick question: I was wondering about naming of the effort.

To me, "Path" and "Query" imply somewhat different (but overlapping)
feature sets:

* Path leaning towards traversal and selection of nodes, similar to
XPath, JsonPath
* Query seems more oriented towards aggregate operations (SQL, XQuery, Jaql)

Others may have different connotations with these terms.

But given the description of goals, it seems to me that the effort
would be more geared for defining a Path language, and not Query
language.

Am I totally off here? I just wonder if naming might lead people to
read too much into the effort; even if actual goals are clearly
listed.

-+ Tatu +-

> > The first drawback is partly true but in complex scenarios, a user with StAX
parser

> > may also need to keep track of states such as nesting levels, parent-child

> > relationships and so on.


> Maybe, but not necessarily, because this information if implicit

> within call stack (except for having to track end markers).

> That is, it's a recursive-descent kind of approach where you know

> where you came from, usually without additional tracking of location.

> Code branches based on constructs encountered.


Yes, it's convenient. But I think it may result in a call stack based processor
instead of a heap based one, right? The former will cause stack overflow issues
in a deep nesting level. Here's a heap based processor for building object graph
with SAX-like interface:

http://code.google.com/p/json-simple/wiki/DecodingExamples#Example_6_-_Build_who\
le_object_graph_on_top_of_SAX-like_content



       ___________________________________________________________
   好玩贺卡等你发，邮箱贺卡全新上线！
http://card.mail.cn.yahoo.com/

I was reading over the StAX specification and BEA provides
licenses to the API, but that license prevents
sublicenses.  This means I as a vendor cannot provide my
own implementation and license that to customers.   So if
I am reading that right what is the point of that
standard?
We at P6R provide JSON and XML tools (amoung others), but
if the standard has restrictions on it then its not a real
standard that we can use.

Mark
P6R, Inc


On Thu, 5 Feb 2009 15:28:41 +0800 (CST)
   Fang Yidong <fangyidong@...> wrote:
> Well, if I am right, the parser in your example is
>essentially a lexer, with slightly higher abstraction.
>
> It's true that it's convenient to control in simple
>case. But in a slightly more complex scenario, such as
>retrieving data in some desired location (for example,
>'/store/book[1]/title' in XPath expression), I don't
>think the code using a SAX(-like) parser is much more
>complex than using a StAX(-like) parser.
>
> Besides easier to pipeline, a SAX(-like) parser requires
>smaller memory footprint and is faster, and the stoppable
>SAX-like interface introduced by JSON.simple avoids the
>drawback that a traditional SAX parser requires the
>entire document to be parsed to get a simple data.
>
> I think different applications require different
>abstraction levels. JSON.simple's stoppable SAX-like
>interface provides a new option to the user. It's your
>choice of adopting it or not.
> 鍙戜欢浜�: Tatu Saloranta <tsaloranta@...>
> 涓婚: Re: [json] Stoppable SAX-like interface for
>streaming input of JSON  text
> 鏀朵欢浜�: json@yahoogroups.com
> 鏃ユ湡: 2009,25,鍛ㄥ洓,1:22涓婂崍
>
>
>
>
>
>
>
>
>
>
>
>            On Tue, Feb 3, 2009 at 8:09 PM, Fang Yidong
><fangyidong@yahoo. com.cn> wrote:
>
>> JSON.simple introduces a simplified and stoppable
>>SAX-like content handler to process JSON text stream.
>>Please take a look if you are interested in it:
>
>>
>
>
>
> If you are interested in application code controlling
>parsing, why not
>
> just use Stax(-like) pull interface? Code example given
>would be quite
>
> a bit simpler with "pull" approach; essentially little
>more than
>
> recursive descent, or with some interfaces, linear
>iteration like:
>
>
>
> ---
>
> JsonParser jp = factory.createJsonP arser(input) ;
>
> JsonToken t;
>
>
>
> while ((t = jp.nextToken( )) != null) {
>
>  if (t == JsonToken.FIELD_ NAME && "id".equals(
>t.getCurrentName ())) {
>
>    break;
>
>  }
>
> }
>
> if (t != null) { // get value for the field
>
>  t = jp.nextToken( );
>
>  System.out.println( "found id, value: "+jp.getText( ));
>
> }
>
> ---
>
>
>
> And you could obviously built simpler abstractions for
>matching on top of this.
>
>
>
> The main benefit of push-interface like SAX is that it
>is easier to
>
> pipeline multiple processing stages. Otherwise it is
>rather cumbersome
>
> and inconvenient way to process data that naturally
>comes in
>
> well-defined and structured order.
>
>
>
> I am asking because oftentimes xml/json/whatever parser
>writers use
>
> SAX-like approaches without knowing that it's only way
>to slice and
>
> dice data, and often not the best.
>
>
>
> -+ Tatu +-
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>      ___________________________________________________________
>  濂界帺璐哄崱绛変綘鍙戯紝閭璐哄崱鍏ㄦ柊涓婄嚎锛�
> http://card.mail.cn.yahoo.com/
>
> [Non-text portions of this message have been removed]
>

-------------------------
Mark Joseph, Ph.D.
President and Secretary
P6R, Inc.
http://www.p6r.com
408-205-0361
Fax: 831-476-7490
Skype: markjoseph_sc
IM: (Yahoo) mjoseph8888
      (AIM) mjoseph8888

On Sat, Feb 7, 2009 at 4:25 PM, Mark Joseph <mark@...> wrote:
> I was reading over the StAX specification and BEA provides
> licenses to the API, but that license prevents
> sublicenses.  This means I as a vendor cannot provide my
> own implementation and license that to customers.   So if

I don't see why you would need a license to implement an API.
Generally licensing governs usage of API itself, distributing it, modifying etc.
None of those are usually needed, because Stax is part of JDK 1.6.
Or you point users to download API jar itself from whoever can provide it.

Also: whatever stax specs download bundle claims is probably incorrect.

But yes, clearly BEA screwed up licensing mentions and other parts.

> I am reading that right what is the point of that
> standard?
> We at P6R provide JSON and XML tools (amoung others), but
> if the standard has restrictions on it then its not a real
> standard that we can use.

Just to be clear: Stax API itself has little to do with Json. It is a
Java xml processing API, and would be of little help for Json. There's
no point in trying to implement it, due to fundamental differences
between xml and json data formats.

But similar style ("pull parsing") is useful.

-+ Tatu +-

On Fri, Feb 6, 2009 at 5:33 PM, Fang Yidong <fangyidong@...> wrote:
>
>> Maybe, but not necessarily, because this information if implicit
>> within call stack (except for having to track end markers).
>
>> That is, it's a recursive-descent kind of approach where you know
>> where you came from, usually without additional tracking of location.
>> Code branches based on constructs encountered.
>
> Yes, it's convenient. But I think it may result in a call stack based
processor instead of a heap based
> one, right? The former will cause stack overflow issues in a deep nesting
level. Here's a heap

Yes, if your document has nesting level of about million or so. :-D
So I don't think that is a practical concern.

If it happens to be, then one can construct explicit stack, similar to
how one has to do it with SAX-like interfaces.

> based processor for building object graph with SAX-like interface:
>
http://code.google.com/p/json-simple/wiki/DecodingExamples#Example_6_-_Build_who\
le_object_graph_on_top_of_SAX-like_content

Right: that builds "poor man's object binding", List/Map/primitive
structure from Json.
Most Json parsers offer that functionality via API, so it need not be
built from low-level components (json.org and others).
Code with pull API would be quite similar, although one could choose
between recursion and iteration with explicit stack.

-+ Tatu +-

Ah I am sorry I was not clear we provide the JSON and XML
tools to C++ users not Java.
http://www.p6r.com/articles/2008/05/22/a-sax-like-parser-for-json/




On Sat, 7 Feb 2009 19:10:21 -0800
   Tatu Saloranta <tsaloranta@...> wrote:
> On Sat, Feb 7, 2009 at 4:25 PM, Mark Joseph
><mark@...> wrote:
>> I was reading over the StAX specification and BEA
>>provides
>> licenses to the API, but that license prevents
>> sublicenses.  This means I as a vendor cannot provide my
>> own implementation and license that to customers.   So
>>if
>
> I don't see why you would need a license to implement an
>API.
> Generally licensing governs usage of API itself,
>distributing it, modifying etc.
> None of those are usually needed, because Stax is part
>of JDK 1.6.
> Or you point users to download API jar itself from
>whoever can provide it.
>
> Also: whatever stax specs download bundle claims is
>probably incorrect.
>
> But yes, clearly BEA screwed up licensing mentions and
>other parts.
>
>> I am reading that right what is the point of that
>> standard?
>> We at P6R provide JSON and XML tools (amoung others),
>>but
>> if the standard has restrictions on it then its not a
>>real
>> standard that we can use.
>
> Just to be clear: Stax API itself has little to do with
>Json. It is a
> Java xml processing API, and would be of little help for
>Json. There's
> no point in trying to implement it, due to fundamental
>differences
> between xml and json data formats.
>
> But similar style ("pull parsing") is useful.
>
> -+ Tatu +-

-------------------------
Mark Joseph, Ph.D.
President and Secretary
P6R, Inc.
http://www.p6r.com
408-205-0361
Fax: 831-476-7490
Skype: markjoseph_sc
IM: (Yahoo) mjoseph8888
      (AIM) mjoseph8888

On Sat, Feb 7, 2009 at 8:35 PM, Mark Joseph <mark@...> wrote:
> Ah I am sorry I was not clear we provide the JSON and XML
> tools to C++ users not Java.
> http://www.p6r.com/articles/2008/05/22/a-sax-like-parser-for-json/

Ok that explains it. I shouldn't have assume it's for Java either.
And it is true that for products that cover both xml and json, it is
advantageous to use same or similar interfaces too. There are some
java libraries that do something similar, such as jettison that
exposes json through java xml interfaces (stax in this case).

-+ Tatu +-

I have updated the JSONRequest proposal, removing the Origin feature.
I have been persuaded that it can encourage bad practice.

http://json.org/JSONRequest.html

--- In json@yahoogroups.com, "Douglas Crockford" <douglas@...> wrote:
>
> I am proposing a new mechanism for doing data transport in Ajax/Comet
> applications. It is called JSONRequest. It is a minimal communications
> facility that can be exempted from the Same Origin Policy.
>
> You can read about it here: http://json.org/JSONRequest.html
>

This question may have been asked previously but a quick search didn't
turn up anything. Why does JSONRequest only support GET/POST and
propose a new CANCEL request? What of PUT and DELETE? I can see how
JSONRequest would be extremely useful for RESTful web services and I'm
curious as to why you are proposing yet another protocol on top of
HTTP when HTTP already provides the necessary verbs and
exceptions/error codes. Wouldn't it be expedient to work within the
existing HTTP specification? My apologies if this has already been
addressed, I've arrived a little bit late to the discussion.

- Dave

On Mon, Feb 16, 2009 at 12:10 AM, pigwin32 <pigwin32@...> wrote:
> --- In json@yahoogroups.com, "Douglas Crockford" <douglas@...> wrote:
>>
>> I am proposing a new mechanism for doing data transport in Ajax/Comet
>> applications. It is called JSONRequest. It is a minimal communications
>> facility that can be exempted from the Same Origin Policy.
>>
>> You can read about it here: http://json.org/JSONRequest.html
>>
>
> This question may have been asked previously but a quick search didn't
> turn up anything. Why does JSONRequest only support GET/POST and
> propose a new CANCEL request? What of PUT and DELETE? I can see how
> JSONRequest would be extremely useful for RESTful web services and I'm
> curious as to why you are proposing yet another protocol on top of
> HTTP when HTTP already provides the necessary verbs and
> exceptions/error codes. Wouldn't it be expedient to work within the
> existing HTTP specification? My apologies if this has already been
> addressed, I've arrived a little bit late to the discussion.

I am not a REST expert, but isn't PUT used to store given payload (in
this case, request in form of Json), and DELETE just ignore payload?
Meaning that such operations via JSONRequest wouldn't make much sense;
after all, you can call PUT with appropriate content type anyway, and
DELETE wouldn't require one.

That is: what would be useful semantics for such operations? (same
could be asked of GET -- but I assume that's for manual testing via
browser)

I guess case could be made that for completeness sake these verbs
should be supported.

-+ Tatu +-

VTD-XML 2.5 is now released. Please go to
https://sourceforge.net/project/showfiles.php?
group_id=110612&package_id=120172&release_id=661376
   to download the latest version.


Changes from Version 2.4 (2/2009)

* Added separate VTD indexing generating and loading (see
http://vtd-xml.sf.net/persistence.html for further info)
* Integrated extended VTD supporting 256 GB doc (In Java only).
* Added duplicateNav() for replicate multiple VTDNav instances sharing
XML, VTD
and LC buffer (availabe in Java and C#).
* Various bug fixes and enhancements

Hello Tatu,

While I was googling JSON.simple usage, I occasionally find
JsonSimpleDriver.java in svn of codehaus, and the benchmark results, comparing
with Jackson and other libraries. I think that it's your work?

The throughput of Jackson is high definitely and I really think Jackson is an
excellent StAX JSON parser.

I am writing to you because I'd like to share some of my opinions on the testing
itself. I know that you haven't pulished the results yet, here's just some
discussions.

Here's my opinions:
1. I think different libraries accept different inputs, such as a byte array, a
string, a inputstream or a reader, so the preparation of such input objects
should be prepared in the warm up stage, not the running.

2. I think the iteration of the resulting graph is not a part of the parser and
should not be put in the run method, but some minor operations can be performed
to verified that the result is a correct one.

3. Could you also help to test the SAX-like interface of JSON.simple in the way
of JacksonDriverStreaming.java did?聽

I think your work will definitely help to improve the qualities of all JSON
libraries.

Thanks.

Yidong Fang




       ___________________________________________________________
   濂界帺璐哄崱绛変綘鍙戯紝閭璐哄崱鍏ㄦ柊涓婄嚎锛�
http://card.mail.cn.yahoo.com/

[Non-text portions of this message have been removed]

Sorry this message is intended to send to Tatu only. My bad.
Since the result is not published yet, let's discuss it privately.












             Hello Tatu,



While I was googling JSON.simple usage, I occasionally find JsonSimpleDriver.
java in svn of codehaus, and the benchmark results, comparing with Jackson and
other libraries. I think that it's your work?



The throughput of Jackson is high definitely and I really think Jackson is an
excellent StAX JSON parser.



I am writing to you because I'd like to share some of my opinions on the testing
itself. I know that you haven't pulished the results yet, here's just some
discussions.



Here's my opinions:

1. I think different libraries accept different inputs, such as a byte array, a
string, a inputstream or a reader, so the preparation of such input objects
should be prepared in the warm up stage, not the running.



2. I think the iteration of the resulting graph is not a part of the parser and
should not be put in the run method, but some minor operations can be performed
to verified that the result is a correct one.



3. Could you also help to test the SAX-like interface of JSON.simple in the way
of JacksonDriverStream ing.java did?聽



I think your work will definitely help to improve the qualities of all JSON
libraries.



Thanks.



Yidong Fang




















       ___________________________________________________________
   濂界帺璐哄崱绛変綘鍙戯紝閭璐哄崱鍏ㄦ柊涓婄嚎锛�
http://card.mail.cn.yahoo.com/

[Non-text portions of this message have been removed]

On Thu, Feb 19, 2009 at 11:08 PM, Fang Yidong <fangyidong@...> wrote:
> Hello Tatu,
>
> While I was googling JSON.simple usage, I occasionally find
JsonSimpleDriver.java in svn of codehaus, and the benchmark results, comparing
with Jackson and other libraries. I think that it's your work?

Hi there! Yes, that is true.

>
> The throughput of Jackson is high definitely and I really think Jackson is an
excellent StAX JSON parser.
>
> I am writing to you because I'd like to share some of my opinions on the
testing itself. I know that you haven't pulished the results yet, here's just
some discussions.

Sure, thank you for your input. I hadn't yet published it, but it is
of course public from the blog.

>
> Here's my opinions:
> 1. I think different libraries accept different inputs, such as a byte array,
a string, a inputstream or a reader, so the preparation of such input objects
should be prepared in the warm up stage, not the running.

Hmmh. I don't think I fully agree, but I agree in that it depends on use case.

For me, the use case is always getting Json from an external source
(network request, from file system), and hence input is naturally a
stream of bytes. So if a library just takes a String, it must be
constructed from bytes.
I did give shortcuts for some parsers so that there was no need to
read from the stream, so in a way some parsers benefited from the
setup (from my perspective).

It is possible that other use cases could be ones where a String would
be needed anyway (like perhaps reading it from a DB), but for me this
is not the case.

> 2. I think the iteration of the resulting graph is not a part of the parser
and should not be put in the run method, but some minor operations can be
performed to verified that the result is a correct one.

I think needs to be, because otherwise you can not do anything of use;
and parsers could optimize away some processing.
I don't think results would differ a lot though, most parsers do
handle all the data independent of traversal. I can probably make this
configurable to see what the difference is.

> 3. Could you also help to test the SAX-like interface of JSON.simple in the
way of JacksonDriverStreaming.java did?

Ah yes. Apologies for not trying it out -- I forgot that json.simple
does in fact have a streaming interface. I can add that, and make it
the main interface.

> I think your work will definitely help to improve the qualities of all JSON
libraries.

Thank you, and thanks for feedback!

-+ Tatu +-

On Thu, Feb 19, 2009 at 11:14 PM, Fang Yidong <fangyidong@...> wrote:
> Sorry this message is intended to send to Tatu only. My bad.
> Since the result is not published yet, let's discuss it privately.

(took the discussion offline, but if anyone is interested in what
results we are discussing, those are at:

http://www.cowtowncoder.com/blog/archives/2009/02/entry_204.html

I will be making some updates; and if any owners of other packages
have suggestions, let me know as well. Benchmarks are hard to do well
with different use cases and emphasis, so at least I want to document
presumptions and biases.

-+ Tatu +-

Dear all,

I am pleased to announce the publication of a new open source product,
jyson, a pure java JSON codec for jython 2.5.

You can access source code, documentation, etc, for jyson here

http://jyson.xhaus.com

If you're wondering what benefit using jyson brings over other
available JSON codecs, please see the Jyson FAQ

http://opensource.xhaus.com/projects/jyson/wiki/JysonFaq

Regards,

Alan Kennedy.

Hi.  I'm using JSON with JQGrid in an inquiry program.  I'm building a JSON
string with the server-side program, and the string contains an HTML hyperlink
tag.  Thus, the string should look something like this:

{"rows":[
{"divisionNumber":"4","divisionName":"<a
href=\"inv0004?productGroup=\"WFB\"&division=\"4\">MONTGOMERY - NEW
STEEL</a>","onHandWeight":"336660.28","percentOfTotal":"3.74","perLbCost":".45",\
"avgMonthlyWeight":"223615","monthsSupply":"3.37","turnoverRate":"4.69"}]}

This string works enough.  The link show up, but when I click on it, it takes me
to a URL of "inv0004?productGroup=".  Is there another character that needs to
be escaped?

It's valid JSON but not HTML. You're effectively closing the href attribute with
the first \" in the URL.

First off, it's is going to be an issue because you're nesting unescaped quotes
in a quoted string.  To do that in a JSON string you'd need an extra backslash
to escape the backslash. '\"' becomes '\\"'.  But in HTML, you'd actually need
to encode a quote with '"e;'

But your next problem is URLs aren't supposed to have unescaped quotes.  You
should instead replace '\"' with '%22'.

It looks like the root of your issue is that you're using string concatenation
to build up your result rather than proper encoding.

--- In json@yahoogroups.com, "violinssoundcool" <violinssoundcool@...> wrote:
>
> Hi.  I'm using JSON with JQGrid in an inquiry program.  I'm building a JSON
string with the server-side program, and the string contains an HTML hyperlink
tag.  Thus, the string should look something like this:
>
> {"rows":[
> {"divisionNumber":"4","divisionName":"<a
href=\"inv0004?productGroup=\"WFB\"&division=\"4\">MONTGOMERY - NEW
STEEL</a>","onHandWeight":"336660.28","percentOfTotal":"3.74","perLbCost":".45",\
"avgMonthlyWeight":"223615","monthsSupply":"3.37","turnoverRate":"4.69"}]}
>
> This string works enough.  The link show up, but when I click on it, it takes
me to a URL of "inv0004?productGroup=".  Is there another character that needs
to be escaped?
>

On Mon, Mar 23, 2009 at 12:53 PM, Stephen M. McKamey <jsonml@...> wrote:
...
> It looks like the root of your issue is that you're using string concatenation
to build up your result rather than proper encoding.

This does seem to be the case, and if so, I agree that it's time to
use a proper json encoder.

Just as with xml, one should never use plain old string concatenation
for producing content. Sometimes it's necessary to create/edit json
(xml) by hand, but if it's done programmatically, proper tools should
be used.
Same goes for parsing, too, but most developers pick up that side more
easily. :)

-+ Tatu +-

You're totally right.  Wow.  I honestly can't believe I missed that.  I'm no
genius, but I didn't think I was a complete idiot until now.  I removed the
unnecessary quotes, and it works great now.  So, in the end, it looks like this:

{"rows":[
{"divisionNumber":"4","divisionName":"<a
href=\"inv0004?productGroup=WFB&division=4\">MONTGOMERY - NEW
STEEL</a>",...}]}

By the way, thanks for taking the time to respond to my post.

--- In json@yahoogroups.com, "violinssoundcool" <violinssoundcool@...> wrote:
>
> Hi.  I'm using JSON with JQGrid in an inquiry program.  I'm building a JSON
string with the server-side program, and the string contains an HTML hyperlink
tag.  Thus, the string should look something like this:
>
> {"rows":[
> {"divisionNumber":"4","divisionName":"<a
href=\"inv0004?productGroup=\"WFB\"&division=\"4\">MONTGOMERY - NEW
STEEL</a>","onHandWeight":"336660.28","percentOfTotal":"3.74","perLbCost":".45",\
"avgMonthlyWeight":"223615","monthsSupply":"3.37","turnoverRate":"4.69"}]}
>
> This string works enough.  The link show up, but when I click on it, it takes
me to a URL of "inv0004?productGroup=".  Is there another character that needs
to be escaped?


The string appears to be properly escaped from JSON's perspective. Your problem
lies elsewhere.

I think it is extremely unwise to pass strings received from the network into
innerHTML. That pattern is insecure.

Sorry typo, that HTML quote entity should not have the e on the end: "

--- In json@yahoogroups.com, "Stephen M. McKamey" <jsonml@...> wrote:
>
> It's valid JSON but not HTML. You're effectively closing the href attribute
with the first \" in the URL.
>
> First off, it's is going to be an issue because you're nesting unescaped
quotes in a quoted string.  To do that in a JSON string you'd need an extra
backslash to escape the backslash. '\"' becomes '\\"'.  But in HTML, you'd
actually need to encode a quote with '"e;'
>
> But your next problem is URLs aren't supposed to have unescaped quotes.  You
should instead replace '\"' with '%22'.
>
> It looks like the root of your issue is that you're using string concatenation
to build up your result rather than proper encoding.
>
> --- In json@yahoogroups.com, "violinssoundcool" <violinssoundcool@> wrote:
> >
> > Hi.  I'm using JSON with JQGrid in an inquiry program.  I'm building a JSON
string with the server-side program, and the string contains an HTML hyperlink
tag.  Thus, the string should look something like this:
> >
> > {"rows":[
> > {"divisionNumber":"4","divisionName":"<a
href=\"inv0004?productGroup=\"WFB\"&division=\"4\">MONTGOMERY - NEW
STEEL</a>","onHandWeight":"336660.28","percentOfTotal":"3.74","perLbCost":".45",\
"avgMonthlyWeight":"223615","monthsSupply":"3.37","turnoverRate":"4.69"}]}
> >
> > This string works enough.  The link show up, but when I click on it, it
takes me to a URL of "inv0004?productGroup=".  Is there another character that
needs to be escaped?
> >
>

I'd like to solicit feedback on this minimal (but powerful) template
language I wrote:

http://code.google.com/p/json-template/

It's meant to be a language-independent specification for formatting JSON
data, as JSON is a language-independent data format.  Currently there are
Python, JavaScript, and Java implementations.

This article explains that it's meant to be the *complement* of JSON -- a
declarative/functional way to turn machine-readable data (JSON) into
human-readable data:

http://json-template.googlecode.com/svn/trunk/doc/Introducing-JSON-Template.html

I'm also interested in the "mashups" or "web service coordination" problem,
and this suggested architecture lends itself well to the problem.

The project was just made public last week, so it's fairly new.  But I think
the language has withstood the test of a decent amount of public scrutiny,
and the code is solid and well-tested.  I would appreciate any feedback,
either here or in the group: http://groups.google.com/group/json-template

thanks,
Andy


[Non-text portions of this message have been removed]

I have recently adapted a VBA Excel macro that parses multidimensional
JSON data (ie it deals with arrays and objects in the data), fixed some
parsing bugs and sped it up with a string builder class, and am
publishing this under the BSD license on my website:

http://www.ediy.co.nz/vbjson-json-parser-library-in-vb6-xidc55680.html

This should be useful for other Visual Basic 6 programmers. I would
appreciate any feedback or suggestions for improvements.




[Non-text portions of this message have been removed]

One thing that is currently missing (AFAIK) from JSON stack is the
equivalent of useful parts of W3C Schema ("xml schema").
I know there is a JSON Schema effort underway already, but if I
understand things correctly, its focus on validation aspects, and not
data typing or modeling .
Although w3c Schema can be used for that (as can DTDs and RelaxNG,
latter of which does this better), to me the main value of Schema is
as abstract data typing/model.

What I mean by this is that the only real advantage of, say, SOAP over
similar simpler JSON+Rest approach is that of having language-agnostic
data model that can be used for full object binding and serialization;
including code generation if need be. You can (painfully) define such
model, and then generate or bind conforming data to objects.
This helps in interoperability as you can describe data content in
language+platform independent way, but with ability to get specific
bindings reliably.
With JSON you can already do data mapping/binding quite well, except
for one area of problems: that of handling polymorphism (inheritance).
Although that can be worked around on language-by-language basis
(adding "class" element in Maps), there is no generic way to do it.

Another thing that I feel most existing Schema languages get wrong is
the false goal of having to be expressed in format being described
(XML schema written in xml). Instead, notation absolutely should be a
DSL: right tool for the job. This is especially crucial for JSON
because of its simplicity: trying to shoehorn a definition language in
JSON seems unnecessarily cumbersome.
It is ok to have a one-to-one mapping between optimal DSL and JSON (or
xml) if need be -- RelaxNG shows a good way of doing that with its
compact (non-xml) notation, and equivalent XML serialization.

So I would be interested in finding a solution for this problem -- it
seems like the last missing selling point for JSON-WS, to be used for
the use case of external entities communication over such an interface
(less needed for internal integration where more concrete interfaces,
client libs etc, can be used).

One more thing: defining such a notation might not be extremely
difficult -- since validation is NOT the main focus, range & size
limitations could be omitted, or kept very simple; more important is
JSON-primitive/object-inheritance data-typing and structural
definitions.
Validation aspects can be handled separately; or as an add-on.

-+ Tatu +-

Tatu Saloranta scripsit:

> One thing that is currently missing (AFAIK) from JSON stack is the
> equivalent of useful parts of W3C Schema ("xml schema").

Very interesting.  I've been thinking about this a bit, and here's a
sketch of what I've come up with.  I'm assuming that the target language
looks like C++, Java, or C#: that is, objects have statically typed
members, there exists a subtyping relationship between types such that
a subtype has all the members of its supertypes, and Null (the type of
null) is a subtype of every object type.  Note that it doesn't matter
whether the language provides single or multiple inheritance.

In XML, data binding is based on the element name (for DTD-based systems)
or on the element name plus the types of ancestor elements (for W3C XML
Schema based systems).  JSON objects have no convenient analogue of the
element name, which is not only in a distinguished place in the element,
but is up front, making data binding on the fly practical.  We could just
require a "name" key in each object, but that would be IMHO against the
spirit of JSON.

So let's instead write down for each JSON object type a predicate that
identifies it.  Here's a tentative list of primitive predicates:

         isNull(keyName)
         isUndefined(keyName)
         isBoolean(keyName)
         isTrue(keyName)
         isFalse(keyName)
         isNumeric(keyName)
         hasNumericValue(keyName, number)
         hasStringValue(keyName, number)
         hasParentTypes(typename, ...)

The hasParentTypes predicate accepts an arbitrary number of type names,
which are in order the type of the parent object, the grandparent object, ....
This predicate is required for WXS-equivalent capability; omitting
it gives DTD-equivalent capability.  In addition, the usual and, or,
and not operators can be applied to construct complex predicates from
these primitives.  The type-assignment engine would have to validate
that no object can be assigned to more than one independent type.

In addition to the predicate, each type also has a set of key->type
mappings, which will be validated during type assignment.  The type in
such a mapping can be a primitive type (boolean, number), an object
type, an array type, or undefined.  So if a type specifies that key
x is boolean, then it must be so.  If it specifies that key y is of
type z, then we make sure the value of key y matches the predicate
for type z or any of the subtypes of z, or is null.  Arrays come in
four flavors: general arrays, boolean arrays, numeric arrays, and
arrays-of-object-type-x.  Undefined means that the key-value pair is
not represented in the object model.

We must have a lattice of subtype-supertype mappings.  For convenience,
it's probably best to specify in the mapping language one or more
supertypes for each type, with the understanding that the key->type
mappings of the supertype(s) are also assumed for the subtype.  This is
just a notational convenience, but OO programmers expect it.  In addition,
the subtype predicate as written is logically ANDed with the supertype
predicate(s) to form the effective predicate for the subtype.  This
hopefully preserves the Liskov substitution principle for the bound types.
This is the only exception to the rule that type predicates must be
independent.

Does this look like what you had in mind?

--
Her he asked if O'Hare Doctor tidings sent from far     John Cowan
coast and she with grameful sigh him answered that      http://ccil.org/~cowan
O'Hare Doctor in heaven was. Sad was the man that word  cowan@...
to hear that him so heavied in bowels ruthful. All
she there told him, ruing death for friend so young,    James Joyce, Ulysses
algate sore unwilling God's rightwiseness to withsay.   "Oxen of the Sun"

I thought this is interesting; as per these entries:

http://docs.amazonwebservices.com/AWSSimpleQueueService/2009-02-01/SQSDeveloperG\
uide/index.html?AccessPolicyLanguage.html
http://cloudsecurity.org/2009/04/09/amazon-aws-introduces-new-access-policy-lang\
uage-sqs-today/

it looks like Amazon is defining its new access policy language (to be
used for most/all AWS services, and starting with the Simple Queue
Service, if I understand this correctly) to be based on JSON.
Neat.

-+ Tatu +-

Tatu,

I don't really understand XML schema or WSDL (which I think you're
referring to), so I could be quite off target here ...

but maybe Google Protocol Buffers fit the problem?

http://code.google.com/apis/protocolbuffers/docs/proto.html

JD



On Apr 25, 2009, at 8:58 AM, Tatu Saloranta wrote:

> One thing that is currently missing (AFAIK) from JSON stack is the
> equivalent of useful parts of W3C Schema ("xml schema").
> I know there is a JSON Schema effort underway already, but if I
> understand things correctly, its focus on validation aspects, and not
> data typing or modeling .
> Although w3c Schema can be used for that (as can DTDs and RelaxNG,
> latter of which does this better), to me the main value of Schema is
> as abstract data typing/model.
>
> What I mean by this is that the only real advantage of, say, SOAP over
> similar simpler JSON+Rest approach is that of having language-agnostic
> data model that can be used for full object binding and serialization;
> including code generation if need be. You can (painfully) define such
> model, and then generate or bind conforming data to objects.
> This helps in interoperability as you can describe data content in
> language+platform independent way, but with ability to get specific
> bindings reliably.
> With JSON you can already do data mapping/binding quite well, except
> for one area of problems: that of handling polymorphism (inheritance).
> Although that can be worked around on language-by-language basis
> (adding "class" element in Maps), there is no generic way to do it.
>
> Another thing that I feel most existing Schema languages get wrong is
> the false goal of having to be expressed in format being described
> (XML schema written in xml). Instead, notation absolutely should be a
> DSL: right tool for the job. This is especially crucial for JSON
> because of its simplicity: trying to shoehorn a definition language in
> JSON seems unnecessarily cumbersome.
> It is ok to have a one-to-one mapping between optimal DSL and JSON (or
> xml) if need be -- RelaxNG shows a good way of doing that with its
> compact (non-xml) notation, and equivalent XML serialization.
>
> So I would be interested in finding a solution for this problem -- it
> seems like the last missing selling point for JSON-WS, to be used for
> the use case of external entities communication over such an interface
> (less needed for internal integration where more concrete interfaces,
> client libs etc, can be used).
>
> One more thing: defining such a notation might not be extremely
> difficult -- since validation is NOT the main focus, range & size
> limitations could be omitted, or kept very simple; more important is
> JSON-primitive/object-inheritance data-typing and structural
> definitions.
> Validation aspects can be handled separately; or as an add-on.
>
> -+ Tatu +-
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

On Mon, Apr 27, 2009 at 10:14 AM, John Cowan <cowan@...> wrote:
> Tatu Saloranta scripsit:
>
>> One thing that is currently missing (AFAIK) from JSON stack is the
>> equivalent of useful parts of W3C Schema ("xml schema").
>
> Very interesting.  I've been thinking about this a bit, and here's a
> sketch of what I've come up with.  I'm assuming that the target language
> looks like C++, Java, or C#: that is, objects have statically typed

Exactly, since more dynamic languages can use "duck typing", or more
loose conversions. It's static languages that need extra help.

> members, there exists a subtyping relationship between types such that
> a subtype has all the members of its supertypes, and Null (the type of
> null) is a subtype of every object type.  Note that it doesn't matter

Yes. I don't think nulls should be typed; and probably only limitation there

> whether the language provides single or multiple inheritance.

Hmmh. I have to think about that a bit -- you may be right. I have
only considered single-inheritance case so far.

> In XML, data binding is based on the element name (for DTD-based systems)
> or on the element name plus the types of ancestor elements (for W3C XML
> Schema based systems).  JSON objects have no convenient analogue of the
> element name, which is not only in a distinguished place in the

True. But part of it is just "off by one" problem -- there's no name
at root level, but beyond this, there are field names that can be
mapped to schema contextually.

One thing I forgot to mention is that I am effectively using Java
objects as the schema: and there get/set methods and fields do have
names that you can match to JSON object property names (with
overriding by annotations or other configuration).
Same can not be done with explicit schema however, problem has to be
resolved differently like you explain.

> but is up front, making data binding on the fly practical.  We could just
> require a "name" key in each object, but that would be IMHO against the
> spirit of JSON.

Agreed, that seems wrong.

> So let's instead write down for each JSON object type a predicate that
> identifies it.  Here's a tentative list of primitive predicates:
>
>        isNull(keyName)

Would this mean it has to be null, or that it is nullable (allowsNull)?

>        isUndefined(keyName)

does this mean "any type"? Sort of fallback, xml "any" type.

>        isBoolean(keyName)
>        isTrue(keyName)
>        isFalse(keyName)
>        isNumeric(keyName)

perhaps also isInteger/integral?

>        hasNumericValue(keyName, number)
>        hasStringValue(keyName, number)

(string instead of 'number')

>        hasParentTypes(typename, ...)
>
> The hasParentTypes predicate accepts an arbitrary number of type names,
> which are in order the type of the parent object, the grandparent object, ....
> This predicate is required for WXS-equivalent capability; omitting
> it gives DTD-equivalent capability.  In addition, the usual and, or,
> and not operators can be applied to construct complex predicates from
> these primitives.  The type-assignment engine would have to validate
> that no object can be assigned to more than one independent type.

Yes.

> In addition to the predicate, each type also has a set of key->type
> mappings, which will be validated during type assignment.  The type in
...
> it's probably best to specify in the mapping language one or more
> supertypes for each type, with the understanding that the key->type
> mappings of the supertype(s) are also assumed for the subtype.  This is
> just a notational convenience, but OO programmers expect it.  In addition,
> the subtype predicate as written is logically ANDed with the supertype
> predicate(s) to form the effective predicate for the subtype.  This
> hopefully preserves the Liskov substitution principle for the bound types.
> This is the only exception to the rule that type predicates must be
> independent.
>
> Does this look like what you had in mind?

It sounds good so far, although I need to read this with more thought.
You have thought more deeply about hard problems than I have, I think. :)

Some additional constraints, compared to other schema work that I
thought of were:

- Not allowing union types that are not mappable to OO ("value can be
either an array or boolean"): this just means there are legal JSON
constructs for which no strict can be defined.
   This is in line with the idea of "schema" being geared towards
mapping to/from OO instances, not for constraining general JSON
content.

-+ Tatu +-