On Mon, Mar 23, 2009 at 12:53 PM, Stephen M. McKamey <jsonml@...> wrote:
...
> It looks like the root of your issue is that you're using string concatenation
to build up your result rather than proper encoding.

This does seem to be the case, and if so, I agree that it's time to
use a proper json encoder.

Just as with xml, one should never use plain old string concatenation
for producing content. Sometimes it's necessary to create/edit json
(xml) by hand, but if it's done programmatically, proper tools should
be used.
Same goes for parsing, too, but most developers pick up that side more
easily. :)

-+ Tatu +-

You're totally right.  Wow.  I honestly can't believe I missed that.  I'm no
genius, but I didn't think I was a complete idiot until now.  I removed the
unnecessary quotes, and it works great now.  So, in the end, it looks like this:

{"rows":[
{"divisionNumber":"4","divisionName":"<a
href=\"inv0004?productGroup=WFB&division=4\">MONTGOMERY - NEW
STEEL</a>",...}]}

By the way, thanks for taking the time to respond to my post.

--- In json@yahoogroups.com, "violinssoundcool" <violinssoundcool@...> wrote:
>
> Hi.  I'm using JSON with JQGrid in an inquiry program.  I'm building a JSON
string with the server-side program, and the string contains an HTML hyperlink
tag.  Thus, the string should look something like this:
>
> {"rows":[
> {"divisionNumber":"4","divisionName":"<a
href=\"inv0004?productGroup=\"WFB\"&division=\"4\">MONTGOMERY - NEW
STEEL</a>","onHandWeight":"336660.28","percentOfTotal":"3.74","perLbCost":".45",\
"avgMonthlyWeight":"223615","monthsSupply":"3.37","turnoverRate":"4.69"}]}
>
> This string works enough.  The link show up, but when I click on it, it takes
me to a URL of "inv0004?productGroup=".  Is there another character that needs
to be escaped?


The string appears to be properly escaped from JSON's perspective. Your problem
lies elsewhere.

I think it is extremely unwise to pass strings received from the network into
innerHTML. That pattern is insecure.

Sorry typo, that HTML quote entity should not have the e on the end: "

--- In json@yahoogroups.com, "Stephen M. McKamey" <jsonml@...> wrote:
>
> It's valid JSON but not HTML. You're effectively closing the href attribute
with the first \" in the URL.
>
> First off, it's is going to be an issue because you're nesting unescaped
quotes in a quoted string.  To do that in a JSON string you'd need an extra
backslash to escape the backslash. '\"' becomes '\\"'.  But in HTML, you'd
actually need to encode a quote with '"e;'
>
> But your next problem is URLs aren't supposed to have unescaped quotes.  You
should instead replace '\"' with '%22'.
>
> It looks like the root of your issue is that you're using string concatenation
to build up your result rather than proper encoding.
>
> --- In json@yahoogroups.com, "violinssoundcool" <violinssoundcool@> wrote:
> >
> > Hi.  I'm using JSON with JQGrid in an inquiry program.  I'm building a JSON
string with the server-side program, and the string contains an HTML hyperlink
tag.  Thus, the string should look something like this:
> >
> > {"rows":[
> > {"divisionNumber":"4","divisionName":"<a
href=\"inv0004?productGroup=\"WFB\"&division=\"4\">MONTGOMERY - NEW
STEEL</a>","onHandWeight":"336660.28","percentOfTotal":"3.74","perLbCost":".45",\
"avgMonthlyWeight":"223615","monthsSupply":"3.37","turnoverRate":"4.69"}]}
> >
> > This string works enough.  The link show up, but when I click on it, it
takes me to a URL of "inv0004?productGroup=".  Is there another character that
needs to be escaped?
> >
>

I'd like to solicit feedback on this minimal (but powerful) template
language I wrote:

http://code.google.com/p/json-template/

It's meant to be a language-independent specification for formatting JSON
data, as JSON is a language-independent data format.  Currently there are
Python, JavaScript, and Java implementations.

This article explains that it's meant to be the *complement* of JSON -- a
declarative/functional way to turn machine-readable data (JSON) into
human-readable data:

http://json-template.googlecode.com/svn/trunk/doc/Introducing-JSON-Template.html

I'm also interested in the "mashups" or "web service coordination" problem,
and this suggested architecture lends itself well to the problem.

The project was just made public last week, so it's fairly new.  But I think
the language has withstood the test of a decent amount of public scrutiny,
and the code is solid and well-tested.  I would appreciate any feedback,
either here or in the group: http://groups.google.com/group/json-template

thanks,
Andy


[Non-text portions of this message have been removed]

I have recently adapted a VBA Excel macro that parses multidimensional
JSON data (ie it deals with arrays and objects in the data), fixed some
parsing bugs and sped it up with a string builder class, and am
publishing this under the BSD license on my website:

http://www.ediy.co.nz/vbjson-json-parser-library-in-vb6-xidc55680.html

This should be useful for other Visual Basic 6 programmers. I would
appreciate any feedback or suggestions for improvements.




[Non-text portions of this message have been removed]

One thing that is currently missing (AFAIK) from JSON stack is the
equivalent of useful parts of W3C Schema ("xml schema").
I know there is a JSON Schema effort underway already, but if I
understand things correctly, its focus on validation aspects, and not
data typing or modeling .
Although w3c Schema can be used for that (as can DTDs and RelaxNG,
latter of which does this better), to me the main value of Schema is
as abstract data typing/model.

What I mean by this is that the only real advantage of, say, SOAP over
similar simpler JSON+Rest approach is that of having language-agnostic
data model that can be used for full object binding and serialization;
including code generation if need be. You can (painfully) define such
model, and then generate or bind conforming data to objects.
This helps in interoperability as you can describe data content in
language+platform independent way, but with ability to get specific
bindings reliably.
With JSON you can already do data mapping/binding quite well, except
for one area of problems: that of handling polymorphism (inheritance).
Although that can be worked around on language-by-language basis
(adding "class" element in Maps), there is no generic way to do it.

Another thing that I feel most existing Schema languages get wrong is
the false goal of having to be expressed in format being described
(XML schema written in xml). Instead, notation absolutely should be a
DSL: right tool for the job. This is especially crucial for JSON
because of its simplicity: trying to shoehorn a definition language in
JSON seems unnecessarily cumbersome.
It is ok to have a one-to-one mapping between optimal DSL and JSON (or
xml) if need be -- RelaxNG shows a good way of doing that with its
compact (non-xml) notation, and equivalent XML serialization.

So I would be interested in finding a solution for this problem -- it
seems like the last missing selling point for JSON-WS, to be used for
the use case of external entities communication over such an interface
(less needed for internal integration where more concrete interfaces,
client libs etc, can be used).

One more thing: defining such a notation might not be extremely
difficult -- since validation is NOT the main focus, range & size
limitations could be omitted, or kept very simple; more important is
JSON-primitive/object-inheritance data-typing and structural
definitions.
Validation aspects can be handled separately; or as an add-on.

-+ Tatu +-

Tatu Saloranta scripsit:

> One thing that is currently missing (AFAIK) from JSON stack is the
> equivalent of useful parts of W3C Schema ("xml schema").

Very interesting.  I've been thinking about this a bit, and here's a
sketch of what I've come up with.  I'm assuming that the target language
looks like C++, Java, or C#: that is, objects have statically typed
members, there exists a subtyping relationship between types such that
a subtype has all the members of its supertypes, and Null (the type of
null) is a subtype of every object type.  Note that it doesn't matter
whether the language provides single or multiple inheritance.

In XML, data binding is based on the element name (for DTD-based systems)
or on the element name plus the types of ancestor elements (for W3C XML
Schema based systems).  JSON objects have no convenient analogue of the
element name, which is not only in a distinguished place in the element,
but is up front, making data binding on the fly practical.  We could just
require a "name" key in each object, but that would be IMHO against the
spirit of JSON.

So let's instead write down for each JSON object type a predicate that
identifies it.  Here's a tentative list of primitive predicates:

         isNull(keyName)
         isUndefined(keyName)
         isBoolean(keyName)
         isTrue(keyName)
         isFalse(keyName)
         isNumeric(keyName)
         hasNumericValue(keyName, number)
         hasStringValue(keyName, number)
         hasParentTypes(typename, ...)

The hasParentTypes predicate accepts an arbitrary number of type names,
which are in order the type of the parent object, the grandparent object, ....
This predicate is required for WXS-equivalent capability; omitting
it gives DTD-equivalent capability.  In addition, the usual and, or,
and not operators can be applied to construct complex predicates from
these primitives.  The type-assignment engine would have to validate
that no object can be assigned to more than one independent type.

In addition to the predicate, each type also has a set of key->type
mappings, which will be validated during type assignment.  The type in
such a mapping can be a primitive type (boolean, number), an object
type, an array type, or undefined.  So if a type specifies that key
x is boolean, then it must be so.  If it specifies that key y is of
type z, then we make sure the value of key y matches the predicate
for type z or any of the subtypes of z, or is null.  Arrays come in
four flavors: general arrays, boolean arrays, numeric arrays, and
arrays-of-object-type-x.  Undefined means that the key-value pair is
not represented in the object model.

We must have a lattice of subtype-supertype mappings.  For convenience,
it's probably best to specify in the mapping language one or more
supertypes for each type, with the understanding that the key->type
mappings of the supertype(s) are also assumed for the subtype.  This is
just a notational convenience, but OO programmers expect it.  In addition,
the subtype predicate as written is logically ANDed with the supertype
predicate(s) to form the effective predicate for the subtype.  This
hopefully preserves the Liskov substitution principle for the bound types.
This is the only exception to the rule that type predicates must be
independent.

Does this look like what you had in mind?

--
Her he asked if O'Hare Doctor tidings sent from far     John Cowan
coast and she with grameful sigh him answered that      http://ccil.org/~cowan
O'Hare Doctor in heaven was. Sad was the man that word  cowan@...
to hear that him so heavied in bowels ruthful. All
she there told him, ruing death for friend so young,    James Joyce, Ulysses
algate sore unwilling God's rightwiseness to withsay.   "Oxen of the Sun"

I thought this is interesting; as per these entries:

http://docs.amazonwebservices.com/AWSSimpleQueueService/2009-02-01/SQSDeveloperG\
uide/index.html?AccessPolicyLanguage.html
http://cloudsecurity.org/2009/04/09/amazon-aws-introduces-new-access-policy-lang\
uage-sqs-today/

it looks like Amazon is defining its new access policy language (to be
used for most/all AWS services, and starting with the Simple Queue
Service, if I understand this correctly) to be based on JSON.
Neat.

-+ Tatu +-

Tatu,

I don't really understand XML schema or WSDL (which I think you're
referring to), so I could be quite off target here ...

but maybe Google Protocol Buffers fit the problem?

http://code.google.com/apis/protocolbuffers/docs/proto.html

JD



On Apr 25, 2009, at 8:58 AM, Tatu Saloranta wrote:

> One thing that is currently missing (AFAIK) from JSON stack is the
> equivalent of useful parts of W3C Schema ("xml schema").
> I know there is a JSON Schema effort underway already, but if I
> understand things correctly, its focus on validation aspects, and not
> data typing or modeling .
> Although w3c Schema can be used for that (as can DTDs and RelaxNG,
> latter of which does this better), to me the main value of Schema is
> as abstract data typing/model.
>
> What I mean by this is that the only real advantage of, say, SOAP over
> similar simpler JSON+Rest approach is that of having language-agnostic
> data model that can be used for full object binding and serialization;
> including code generation if need be. You can (painfully) define such
> model, and then generate or bind conforming data to objects.
> This helps in interoperability as you can describe data content in
> language+platform independent way, but with ability to get specific
> bindings reliably.
> With JSON you can already do data mapping/binding quite well, except
> for one area of problems: that of handling polymorphism (inheritance).
> Although that can be worked around on language-by-language basis
> (adding "class" element in Maps), there is no generic way to do it.
>
> Another thing that I feel most existing Schema languages get wrong is
> the false goal of having to be expressed in format being described
> (XML schema written in xml). Instead, notation absolutely should be a
> DSL: right tool for the job. This is especially crucial for JSON
> because of its simplicity: trying to shoehorn a definition language in
> JSON seems unnecessarily cumbersome.
> It is ok to have a one-to-one mapping between optimal DSL and JSON (or
> xml) if need be -- RelaxNG shows a good way of doing that with its
> compact (non-xml) notation, and equivalent XML serialization.
>
> So I would be interested in finding a solution for this problem -- it
> seems like the last missing selling point for JSON-WS, to be used for
> the use case of external entities communication over such an interface
> (less needed for internal integration where more concrete interfaces,
> client libs etc, can be used).
>
> One more thing: defining such a notation might not be extremely
> difficult -- since validation is NOT the main focus, range & size
> limitations could be omitted, or kept very simple; more important is
> JSON-primitive/object-inheritance data-typing and structural
> definitions.
> Validation aspects can be handled separately; or as an add-on.
>
> -+ Tatu +-
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

On Mon, Apr 27, 2009 at 10:14 AM, John Cowan <cowan@...> wrote:
> Tatu Saloranta scripsit:
>
>> One thing that is currently missing (AFAIK) from JSON stack is the
>> equivalent of useful parts of W3C Schema ("xml schema").
>
> Very interesting.  I've been thinking about this a bit, and here's a
> sketch of what I've come up with.  I'm assuming that the target language
> looks like C++, Java, or C#: that is, objects have statically typed

Exactly, since more dynamic languages can use "duck typing", or more
loose conversions. It's static languages that need extra help.

> members, there exists a subtyping relationship between types such that
> a subtype has all the members of its supertypes, and Null (the type of
> null) is a subtype of every object type.  Note that it doesn't matter

Yes. I don't think nulls should be typed; and probably only limitation there

> whether the language provides single or multiple inheritance.

Hmmh. I have to think about that a bit -- you may be right. I have
only considered single-inheritance case so far.

> In XML, data binding is based on the element name (for DTD-based systems)
> or on the element name plus the types of ancestor elements (for W3C XML
> Schema based systems).  JSON objects have no convenient analogue of the
> element name, which is not only in a distinguished place in the

True. But part of it is just "off by one" problem -- there's no name
at root level, but beyond this, there are field names that can be
mapped to schema contextually.

One thing I forgot to mention is that I am effectively using Java
objects as the schema: and there get/set methods and fields do have
names that you can match to JSON object property names (with
overriding by annotations or other configuration).
Same can not be done with explicit schema however, problem has to be
resolved differently like you explain.

> but is up front, making data binding on the fly practical.  We could just
> require a "name" key in each object, but that would be IMHO against the
> spirit of JSON.

Agreed, that seems wrong.

> So let's instead write down for each JSON object type a predicate that
> identifies it.  Here's a tentative list of primitive predicates:
>
>        isNull(keyName)

Would this mean it has to be null, or that it is nullable (allowsNull)?

>        isUndefined(keyName)

does this mean "any type"? Sort of fallback, xml "any" type.

>        isBoolean(keyName)
>        isTrue(keyName)
>        isFalse(keyName)
>        isNumeric(keyName)

perhaps also isInteger/integral?

>        hasNumericValue(keyName, number)
>        hasStringValue(keyName, number)

(string instead of 'number')

>        hasParentTypes(typename, ...)
>
> The hasParentTypes predicate accepts an arbitrary number of type names,
> which are in order the type of the parent object, the grandparent object, ....
> This predicate is required for WXS-equivalent capability; omitting
> it gives DTD-equivalent capability.  In addition, the usual and, or,
> and not operators can be applied to construct complex predicates from
> these primitives.  The type-assignment engine would have to validate
> that no object can be assigned to more than one independent type.

Yes.

> In addition to the predicate, each type also has a set of key->type
> mappings, which will be validated during type assignment.  The type in
...
> it's probably best to specify in the mapping language one or more
> supertypes for each type, with the understanding that the key->type
> mappings of the supertype(s) are also assumed for the subtype.  This is
> just a notational convenience, but OO programmers expect it.  In addition,
> the subtype predicate as written is logically ANDed with the supertype
> predicate(s) to form the effective predicate for the subtype.  This
> hopefully preserves the Liskov substitution principle for the bound types.
> This is the only exception to the rule that type predicates must be
> independent.
>
> Does this look like what you had in mind?

It sounds good so far, although I need to read this with more thought.
You have thought more deeply about hard problems than I have, I think. :)

Some additional constraints, compared to other schema work that I
thought of were:

- Not allowing union types that are not mappable to OO ("value can be
either an array or boolean"): this just means there are legal JSON
constructs for which no strict can be defined.
   This is in line with the idea of "schema" being geared towards
mapping to/from OO instances, not for constraining general JSON
content.

-+ Tatu +-

On Mon, Apr 27, 2009 at 10:36 AM, John David Duncan
<john.david.duncan@...> wrote:
> Tatu,
>
> I don't really understand XML schema or WSDL (which I think you're
> referring to), so I could be quite off target here ...
>
> but maybe Google Protocol Buffers fit the problem?
>
> http://code.google.com/apis/protocolbuffers/docs/proto.html

I don't think it is directly applicable in PB, schema has much more
fundamental role, being mandatory for operating anything, and to
generate code (unless I'm mistaken). So in a way it is a strict
schema-first format. I am hoping for something that allows both
schema- and code-first approaches, somewhat similar to XML Schema in
that respect.

But maybe some aspects of PB might be useful: syntax, other ideas
regarding versioning?

-+ Tatu +-

Tatu Saloranta scripsit:

> >        isNull(keyName)
>
> Would this mean it has to be null, or that it is nullable (allowsNull)?

All these predicates are on the JSON object itself, because they are used
before the object's type is known.  So isNull("foo") means that there
is a key "foo" in the JSON object value is null, and likewise with all
the other primitives.

> >        isUndefined(keyName)
>
> does this mean "any type"? Sort of fallback, xml "any" type.

No, it means that the specified key does not exist.  I use the name
"undefined" for cultural compatibility with JavaScript.  There would be
little point in an isAnyType predicate, because it would match anything
*except* a missing key.

> >        isNumeric(keyName)
>
> perhaps also isInteger/integral?

That's reasonable.  In that case it would be useful to be able to map
keys to integer as well, and perhaps standardized subtypes of integer too.

> - [U]nion types that are not mappable to OO ("value can be
> either an array or boolean"): this just means there are legal JSON
> constructs for which no strict can be defined.

Multiple inheritance is a subset of this (doesn't handle primitives).
If type foobar is a subtype of types foo and bar, then effectively it
is a union of them.

One question: should the predicates and the maps do on-the-fly conversion?
For example, if we specify a predicate of hasNumericValue("foo", 0),
does "foo": "0" match, or do we require "foo": 0?  Likewise, when
mapping, if the map specifies bar->String and the object contains
"bar": 123, does the bar field get "123", or is that an error?
These questions are independent.  I'd favor doing the conversions.

--
You let them out again, Old Man Willow!                 John Cowan
What you be a-thinking of?  You should not be waking!   cowan@...
Eat earth!  Dig deep!  Drink water!  Go to sleep!
Bombadil is talking.                                    http://ccil.org/~cowan

Tatu Saloranta wrote:
>
>
> One thing that is currently missing (AFAIK) from JSON stack is the
> equivalent of useful parts of W3C Schema ("xml schema").
> I know there is a JSON Schema effort underway already, but if I
> understand things correctly, its focus on validation aspects, and not
> data typing or modeling .
> Although w3c Schema can be used for that (as can DTDs and RelaxNG,
> latter of which does this better), to me the main value of Schema is
> as abstract data typing/model.

JSON Schema can certainly be used for that purpose, I am using JSON
Schema for typing in Dojo and Persevere, and not just for validating
existing JSON structures.

> What I mean by this is that the only real advantage of, say, SOAP over
> similar simpler JSON+Rest approach is that of having language-agnostic
> data model that can be used for full object binding and serialization;

I am not sure if this exactly what you are talking, but we have been
discussing using JSON Schema as a way to define hyperlink properties in
JSON structures to provide interoperability in RESTful JSON:
-------------------------------------

> including code generation if need be. You can (painfully) define such
> model, and then generate or bind conforming data to objects.
> This helps in interoperability as you can describe data content in
> language+platform independent way, but with ability to get specific
> bindings reliably.
> With JSON you can already do data mapping/binding quite well, except
> for one area of problems: that of handling polymorphism (inheritance).
> Although that can be worked around on language-by-language basis
> (adding "class" element in Maps), there is no generic way to do it.

The "extends" attribute of JSON Schema is designed specifically to meet
the needs of describing data structures with inheritance. Is this
insufficient for your polymorphic description needs?

> Another thing that I feel most existing Schema languages get wrong is
> the false goal of having to be expressed in format being described
> (XML schema written in xml). Instead, notation absolutely should be a
> DSL: right tool for the job. This is especially crucial for JSON
> because of its simplicity: trying to shoehorn a definition language in
> JSON seems unnecessarily cumbersome.
> It is ok to have a one-to-one mapping between optimal DSL and JSON (or
> xml) if need be -- RelaxNG shows a good way of doing that with its
> compact (non-xml) notation, and equivalent XML serialization.
>
> So I would be interested in finding a solution for this problem -- it
> seems like the last missing selling point for JSON-WS, to be used for
> the use case of external entities communication over such an interface
> (less needed for internal integration where more concrete interfaces,
> client libs etc, can be used).

"JSON-WS" hints more at RPC type communication, which is what SMD is
designed for (which uses JSON Schema):


> One more thing: defining such a notation might not be extremely
> difficult -- since validation is NOT the main focus, range & size
> limitations could be omitted, or kept very simple; more important is
> JSON-primitive/object-inheritance data-typing and structural
> definitions.
> Validation aspects can be handled separately; or as an add-on.

JSON Schema can as simple as the subset you want to use. {} is a valid
schema (although not particularly useful). We have discussed formally
defining a subset of JSON Schema for more traditional data-typing style
constraints, but when after some discussion of how easy it is to ignore
non-relevant parts of the spec, it seemed like kind of a silly exercise.
Of course, if you want to define a non JSON-based data type modeling
system, that is indeed a completely different matter and exercise. Or
maybe I am missing what you are really after.

Kris

On Mon, Apr 27, 2009 at 2:25 PM, Kris Zyp <kriszyp@...> wrote:
>
> Tatu Saloranta wrote:
>>
>> One thing that is currently missing (AFAIK) from JSON stack is the
>> equivalent of useful parts of W3C Schema ("xml schema").
...
>
> JSON Schema can certainly be used for that purpose, I am using JSON
> Schema for typing in Dojo and Persevere, and not just for validating
> existing JSON structures.

Ok. I did notice 'extends' property (but only after sending email),
which when combined with other pieces should allow for defining type
structures?

>> What I mean by this is that the only real advantage of, say, SOAP over
>> similar simpler JSON+Rest approach is that of having language-agnostic
>> data model that can be used for full object binding and serialization;
>
> I am not sure if this exactly what you are talking, but we have been
> discussing using JSON Schema as a way to define hyperlink properties in
> JSON structures to provide interoperability in RESTful JSON:

Basically I am thinking of supporting both schema-first model
(generaring classes/stubs from definition) and code first (annotating
to produce schema).
And for Java, perhaps generate bean-validation annotations, for schema
first case.

Perhaps what would be nice would be more along the lines of providing
more convenient syntax to express already defined json schema model.

...
> The "extends" attribute of JSON Schema is designed specifically to meet
> the needs of describing data structures with inheritance. Is this
> insufficient for your polymorphic description needs?

Probably not -- initially I just missed it. :-/

It does however need to be coupled with some definition of how to
include type identifier in json content ("#type" field? standard name,
or perhaps configurable with schema). Is there something in Json
schema to define this? Or if not, could it be added?
Type is needed to deserialize specific sub-type instance; on
serialization full type is known, but on deserialization only declared
type, and full type needs to be available from json content (can
optimize this out for leaf types).

...
> "JSON-WS" hints more at RPC type communication, which is what SMD is
> designed for (which uses JSON Schema):

True, I should have clarified that. This would be the data/type model
that is useful (and sort of required) for WS, but specifically not the
web service definition itself (no end points, operations etc defined).
Like XML Schema part that Soap/WSDL build on.

...
> JSON Schema can as simple as the subset you want to use. {} is a valid
> schema (although not particularly useful). We have discussed formally
> defining a subset of JSON Schema for more traditional data-typing style
> constraints, but when after some discussion of how easy it is to ignore
> non-relevant parts of the spec, it seemed like kind of a silly exercise.
> Of course, if you want to define a non JSON-based data type modeling
> system, that is indeed a completely different matter and exercise. Or
> maybe I am missing what you are really after.

Ok, thanks. Sub-setting could definitely work. I need to read json
schema proposal with some more thought. I am interested in syntax
part, as well as some limitations on what kinds of schemas would be
allowed -- certain kinds of unions might not be representable on OO
side.

Also: is the use of json as format for schema a fundamental goal? To
me it seems that there are benefits to using more compact and
expressive notation, but not much benefit from using JSON (i.e.
complexity is not within parsing of schema but in processing it).
I don't mind having a json serialization (RelaxNG - like "dual"
model), but really like the idea of having something more optimal for
the domain.

-+ Tatu +-

Hi all,

I have been working on a new JSON Parser for FLEX.
Called JSWOOF.

The Library is fully complient with the JSON spec. and has
been fully optimized for fast encoding and decoding of streams.

you can find it at:
http://www.waynemike.co.uk/jswoof

Is there some advantage to using

"Action": ["SQS:SendMessage","SQS:ReceiveMessage"],

that I cant see?

Can I see an example of code getting keys and values from this (that shows why
it is a good structure)?

Thanks


To: json@yahoogroups.com
From: tsaloranta@...
Date: Mon, 20 Apr 2009 09:21:35 -0700
Subject: [json] JSON being used for Amazon Web Services new access policy
language


























       I thought this is interesting; as per these entries:



http://docs.amazonwebservices.com/AWSSimpleQueueService/2009-02-01/SQSDeveloperG\
uide/index.html?AccessPolicyLanguage.html

http://cloudsecurity.org/2009/04/09/amazon-aws-introduces-new-access-policy-lang\
uage-sqs-today/



it looks like Amazon is defining its new access policy language (to be

used for most/all AWS services, and starting with the Simple Queue

Service, if I understand this correctly) to be based on JSON.

Neat.



-+ Tatu +-




















_________________________________________________________________
Looking for a fresh way to share photos? Get the new Windows Live Messenger.
http://download.live.com/

[Non-text portions of this message have been removed]

On Mon, Apr 27, 2009 at 5:06 PM, Mark Ireland <markincuba@...> wrote:
>
> Is there some advantage to using
>
> "Action": ["SQS:SendMessage","SQS:ReceiveMessage"],
>
> that I cant see?
>
> Can I see an example of code getting keys and values from this (that shows why
it is a good structure)?

That would be a question to ask Amazon Web Services developers...
it may have something to do with their existing systems and formats
(many xml based).

But at low level that's just a field with array value, and access
style depends on json library in use.

-+ Tatu +-

On Tue, Apr 28, 2009 at 1:28 AM, W <wayne.ivor@...> wrote:

> Hi all,
>
> I have been working on a new JSON Parser for FLEX.
> Called JSWOOF.
>
> The Library is fully complient with the JSON spec. and has
> been fully optimized for fast encoding and decoding of streams.


Can you say some more about the way you tested performance against 'corelib'
(presumably as3corelib .92.1)? Apart from purely size, I'm interested in the
characteristics of the JSON structures you used (e.g. depth of nesting,
ratio of objects to arrays, object sizes, whitespace / density, etc.).

--
Martin Cooper


you can find it at:
> http://www.waynemike.co.uk/jswoof
>
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>


[Non-text portions of this message have been removed]

Testing was not done at a functional. For example i did not test my
string-encoding function against core-libs string-encoding function, etc. but
instead tested the overall speed of my encoding/decoding processes against
core-libs.

Strings where one area i found to have a serious impact on the processor, and
this was mainly due to the way the tokenizer was setting flag states. and the
way most parsers handle the strings in general. For example testing for
white-space in JSWOOF has no comparison tests, but instead uses bitwise
operations.

As for the rest of the encoding and decoding mechanism, it is handled
recursively, so a gain once becomes a gain over and over again. Each time that
optimized area gets called.

You asked about testing on arrays, and objects. But these are merely a
collection of sub entities [strings, number, Boolean, etc]. So density would
also only trigger execution of the same optimized sub-entity functions.

Wayne Mike.


--- In json@yahoogroups.com, Martin Cooper <mfncooper@...> wrote:
>
> On Tue, Apr 28, 2009 at 1:28 AM, W <wayne.ivor@...> wrote:
>
> > Hi all,
> >
> > I have been working on a new JSON Parser for FLEX.
> > Called JSWOOF.
> >
> > The Library is fully complient with the JSON spec. and has
> > been fully optimized for fast encoding and decoding of streams.
>
>
> Can you say some more about the way you tested performance against 'corelib'
> (presumably as3corelib .92.1)? Apart from purely size, I'm interested in the
> characteristics of the JSON structures you used (e.g. depth of nesting,
> ratio of objects to arrays, object sizes, whitespace / density, etc.).
>
> --
> Martin Cooper
>
>
> you can find it at:
> > http://www.waynemike.co.uk/jswoof
> >
> >
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>

On Tue, Apr 28, 2009 at 1:33 PM, W <wayne.ivor@...> wrote:

> Testing was not done at a functional. For example i did not test my
> string-encoding function against core-libs string-encoding function, etc.
> but instead tested the overall speed of my encoding/decoding processes
> against core-libs.
>
> Strings where one area i found to have a serious impact on the processor,
> and this was mainly due to the way the tokenizer was setting flag states.
> and the way most parsers handle the strings in general. For example testing
> for white-space in JSWOOF has no comparison tests, but instead uses bitwise
> operations.
>
> As for the rest of the encoding and decoding mechanism, it is handled
> recursively, so a gain once becomes a gain over and over again. Each time
> that optimized area gets called.
>
> You asked about testing on arrays, and objects. But these are merely a
> collection of sub entities [strings, number, Boolean, etc]. So density would
> also only trigger execution of the same optimized sub-entity functions.


The reason I'm asking is because a JSON object with a flat 10,000 name /
value pairs would illustrate different performance characteristics than
10,000 objects with one name / value pair each, one nested within the other,
within the other, etc., all the way down the chain. Other types of
structures would illustrate different performance characteristics again.
That's why just giving raw times and sizes doesn't tell us very much about
what it's faster at and what it might not be quite as fast at.

--
Martin Cooper



> Wayne Mike.
>
> --- In json@yahoogroups.com, Martin Cooper <mfncooper@...> wrote:
> >
> > On Tue, Apr 28, 2009 at 1:28 AM, W <wayne.ivor@...> wrote:
> >
> > > Hi all,
> > >
> > > I have been working on a new JSON Parser for FLEX.
> > > Called JSWOOF.
> > >
> > > The Library is fully complient with the JSON spec. and has
> > > been fully optimized for fast encoding and decoding of streams.
> >
> >
> > Can you say some more about the way you tested performance against
> 'corelib'
> > (presumably as3corelib .92.1)? Apart from purely size, I'm interested in
> the
> > characteristics of the JSON structures you used (e.g. depth of nesting,
> > ratio of objects to arrays, object sizes, whitespace / density, etc.).
> >
> > --
> > Martin Cooper
> >
> >
> > you can find it at:
> > > http://www.waynemike.co.uk/jswoof
> > >
> > >
> > >
> > >
> > >
> > > ------------------------------------
> > >
> > > Yahoo! Groups Links
> > >
> > >
> > >
> > >
> >
> >
> > [Non-text portions of this message have been removed]
> >
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>
>


[Non-text portions of this message have been removed]

Any entity that could be nested had to be tested rigorously as this type
of structuring naturally slows down processing.

The project i originally designed JSWOOF for uses allot of nested arrays
Some times going more than 50 children deep.

Nested objects were also used alot throughout the project, mainly to populate
data-grids. But bear in mind that these grids had twistable rows with a maximum
of 20 twists.

The project started out life using the core-lib library, hence the reason why i
Compared results with this library on the JSWOOF website.

i noticed things slow down using the core-lib library initially when we started
Nesting the JSON we were using. i.e.

fig 1:
------
{
	 "name" :   "value",
	 "name" :   "value",
	 "name" :   array[...]
}

Something similar to the above really made core-lib slow down. It just did not
like all
that white space. And in actual fact there were no carriage returns in the JSON
we
Passed to core-lib as that would cause it to crash, so it only contianed spaces.
JSWOOF
Will however handle fully structured JSON as shown in fig 1 with carriage
returns and tabs included.

So as stated in my previous post, this was the first issue I came accross, and
the way
JSWOOF gets around this is to use bit-wise operations to check character types.

The second thing i noticed was that objects and arrays work in a very similar
fashion
but for conversation sake we will concentrate on arrays only. take a look a fig
2.

fig 2 - typical JSON Array creation.
-----
a) Allocate memory for array.
b) Insert element into array.
c) Move onto next possible element - goto (b).

Now if this were a nested array or object we would have to allocate some space
for the
new node so fig2 would become.

a) Allocate memory for array.
b) Insert element into array.
c) Move onto next possible element
c1)  Element = array (goto a)
		 Element = xxx

As you can see memory allocation is being done over and over again for each sub
array, and this
seriously adds to the processing time. JSWOOF does not take this approach it
instead creates
a small pool of arrays and objects for use by the object parser and array parser
see fig3.

fig3
----
a) get array from pool.
b) insert element into array.
c) move onto next possible element
c1)  element = array (goto a)
		 elemtnt = xxx

Wayne IV Mike.


--- In json@yahoogroups.com, Martin Cooper <mfncooper@...> wrote:
>
> On Tue, Apr 28, 2009 at 1:33 PM, W <wayne.ivor@...> wrote:
>
> > Testing was not done at a functional. For example i did not test my
> > string-encoding function against core-libs string-encoding function, etc.
> > but instead tested the overall speed of my encoding/decoding processes
> > against core-libs.
> >
> > Strings where one area i found to have a serious impact on the processor,
> > and this was mainly due to the way the tokenizer was setting flag states.
> > and the way most parsers handle the strings in general. For example testing
> > for white-space in JSWOOF has no comparison tests, but instead uses bitwise
> > operations.
> >
> > As for the rest of the encoding and decoding mechanism, it is handled
> > recursively, so a gain once becomes a gain over and over again. Each time
> > that optimized area gets called.
> >
> > You asked about testing on arrays, and objects. But these are merely a
> > collection of sub entities [strings, number, Boolean, etc]. So density would
> > also only trigger execution of the same optimized sub-entity functions.
>
>
> The reason I'm asking is because a JSON object with a flat 10,000 name /
> value pairs would illustrate different performance characteristics than
> 10,000 objects with one name / value pair each, one nested within the other,
> within the other, etc., all the way down the chain. Other types of
> structures would illustrate different performance characteristics again.
> That's why just giving raw times and sizes doesn't tell us very much about
> what it's faster at and what it might not be quite as fast at.
>
> --
> Martin Cooper
>
>
>
> > Wayne Mike.
> >
> > --- In json@yahoogroups.com, Martin Cooper <mfncooper@> wrote:
> > >
> > > On Tue, Apr 28, 2009 at 1:28 AM, W <wayne.ivor@> wrote:
> > >
> > > > Hi all,
> > > >
> > > > I have been working on a new JSON Parser for FLEX.
> > > > Called JSWOOF.
> > > >
> > > > The Library is fully complient with the JSON spec. and has
> > > > been fully optimized for fast encoding and decoding of streams.
> > >
> > >
> > > Can you say some more about the way you tested performance against
> > 'corelib'
> > > (presumably as3corelib .92.1)? Apart from purely size, I'm interested in
> > the
> > > characteristics of the JSON structures you used (e.g. depth of nesting,
> > > ratio of objects to arrays, object sizes, whitespace / density, etc.).
> > >
> > > --
> > > Martin Cooper
> > >
> > >
> > > you can find it at:
> > > > http://www.waynemike.co.uk/jswoof
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > ------------------------------------
> > > >
> > > > Yahoo! Groups Links
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > > [Non-text portions of this message have been removed]
> > >
> >
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
> >
>
>
> [Non-text portions of this message have been removed]
>

Hi,

I'm developing a web front-end to our student database and I'd like to
let the users specify what fields they want in their view and be able to
filter and sort the data.

After some thinking I figured out that this all can be expressed by an
SQL query -- and it will end up generating an SQL query on the server.

So I made a JSON syntax like this:
{
"fields":["std_id","code","lastname","firstname"],
"filters":[{"prog":["200A0","200A1"]},{"decision":["A%"]}],
"sort":[{"cote_mix":"desc"}]
}
which would generate an SQL query like this:
SELECT std_id, code, lastname, firstname FROM students
WHERE prog IN ('200A0','200A1') AND decision LIKE 'A%'
ORDER BY cote_mix DESC

But I feel like I'm re-inventing the wheel here...
Is there already a specification for expressing an SQL query in JSON?

Thanks,
GFK's
--
Guillaume Filion
http://guillaume.filion.org/



[Non-text portions of this message have been removed]

I a new at JSON but have been doing stuff like this for 27 years.  (JSON is
just another form of data transportation as when EDI was king over XML 10-25
years ago.)



You should only give your user's the options on a screen of what fields to
choose, not have them write out the fields names themselves. (Is that what
you are doing?)  You would face a big inject attack if you let them write
the SQL fields themselves.



Gene











From: json@yahoogroups.com [mailto:json@yahoogroups.com] On Behalf Of
Guillaume Filion
Sent: Wednesday, April 29, 2009 3:21 PM
To: json@yahoogroups.com
Subject: [json] Expressing SQL queries in JSON








Hi,

I'm developing a web front-end to our student database and I'd like to
let the users specify what fields they want in their view and be able to
filter and sort the data.

After some thinking I figured out that this all can be expressed by an
SQL query -- and it will end up generating an SQL query on the server.

So I made a JSON syntax like this:
{
"fields":["std_id","code","lastname","firstname"],
"filters":[{"prog":["200A0","200A1"]},{"decision":["A%"]}],
"sort":[{"cote_mix":"desc"}]
}
which would generate an SQL query like this:
SELECT std_id, code, lastname, firstname FROM students
WHERE prog IN ('200A0','200A1') AND decision LIKE 'A%'
ORDER BY cote_mix DESC

But I feel like I'm re-inventing the wheel here...
Is there already a specification for expressing an SQL query in JSON?

Thanks,
GFK's
--
Guillaume Filion
http://guillaume.filion.org/

[Non-text portions of this message have been removed]





[Non-text portions of this message have been removed]

Hey everyone,

Just wanted to announce my new ruby C bindings to Yajl creatively called:
yajl-ruby ;)

The sources are up on Github for now at http://github.com/brianmario/yajl-ruby
although I might put it up on RubyForge soon too. Instructions on how to install
and use are in the README.rdoc file.

To my knowledge this is only the second C-based JSON parser for Ruby aside from
the JSON gem that's existed for a while.

In my preliminary parsing benchmarks, yajl-ruby performed on-par (in terms of
time to completion) with the JSON gem on small-medium sized JSON strings (385
bytes to around 25k) but performs significantly better on larger files (100k+).
But as far as memory usage is concerned - it's much more efficient. For example:
parsing a 2.4MB JSON string, ruby was using around 30MB of RAM for yajl-ruby and
almost 80MB for the JSON gem. Using the OSX Instruments development tool, doing
the same operation was showing 13MB compared to 42MB of allocations
respectively.

Parsing as a stream opens up a whole new realm of possibilities for using JSON
to transfer large amounts of data. I have some examples, including
Yajl::HttpStream.get(uri) which will parse the JSON response body *directly off
the socket* for the highest possible efficiency. Or even directly from a
compressed Gzip or Bzip2 stream without the need to decompress to a buffer, then
parse. I have tons of feature ideas around Http and Socket communication in
general. But I'd love to hear yours!

My next big API addition will be streaming JSON string creation using Yajl. Then
I'll work on some examples of using yajl-ruby on both sides of a stream.

Please check it out and give me any/all improvement suggestions and/or feature
requests you might have.

The web_send library provides a concise and expressive API for
interacting with arbitrary JSON resources from the web browser. When
used from the Firebug console, it acts like a command line for your
web server; a great help during development of server-side code. The
same API is also convenient for creating an AJAX user interface to
JSON resources; so code born on the interactive command line migrates
smoothly into application code. This tutorial also links to a live web
page <http://waterken.sourceforge.net/bang/> where you can try out the
library against a simple server-side counter object.

SOME QUICK INTRODUCTORY EXAMPLES

For example, say you've got a brand new server-side object sitting at
a URL like: <https://example.com/myApp/obj123>. All you want to do is
invoke one of its methods, to see what happens. Using the web_send
library from the Firebug console, you could write:

factory = lib.web.getLocation();    // grab the window.location URL
drum = lib.Q.post(factory, 'makeDrum', []);

That code generates the following HTTP request:

POST /myApp/obj123?q=makeDrum HTTP/1.1
Host: example.com
Content-Type: text/plain; charset=UTF-8
Content-Length: 2

[]

The arguments to Q.post() are:

    1. remote reference for the target object
    2. optional argument to add to the query string
    3. optional JSON value for the request body

The makeDrum() method didn't take any arguments. Here's one that does:

lib.Q.post(drum, 'bang', [ 1 ]);

Causing the HTTP request:

POST /myApp/obj456?q=bang HTTP/1.1
Host: example.com
Content-Type: text/plain; charset=UTF-8
Content-Length: 5

[ 1 ]

The target URL in the above request was taken from the HTTP response
to the previous request. For this to work, the web_send library
introduces some conventions for HTTP requests and responses. These
conventions are the least restrictive they can be, while still
supporting the client-side remote reference API. This document
explains these conventions and shows how to use legacy JSON resources
that don't follow the conventions.

... (continue reading at <http://waterken.sourceforge.net/web_send/>)

--
"Waterken News: Capability security on the Web"
http://waterken.sourceforge.net/recent.html

This looks really cool. Have you tried it with existing servers to see
how compatible it is with current implementations of HTTP/JSON servers?
This data interaction aspect of this looks similar to the functionality
in Dojo's JsonRestStore, which can interact with CouchDB and Persevere.
It'd be curious if your browser shell works with Persevere, as Persevere
follows RFC 2616 pretty closely as well as some of the RESTful JSON
conventions discussed here, such as JSON referencing:
http://groups.google.com/group/restful-json
Kris

Tyler Close wrote:
>
>
> The web_send library provides a concise and expressive API for
> interacting with arbitrary JSON resources from the web browser. When
> used from the Firebug console, it acts like a command line for your
> web server; a great help during development of server-side code. The
> same API is also convenient for creating an AJAX user interface to
> JSON resources; so code born on the interactive command line migrates
> smoothly into application code. This tutorial also links to a live web
> page <http://waterken.sourceforge.net/bang/
> <http://waterken.sourceforge.net/bang/>> where you can try out the
> library against a simple server-side counter object.
>
> SOME QUICK INTRODUCTORY EXAMPLES
>
> For example, say you've got a brand new server-side object sitting at
> a URL like: <https://example.com/myApp/obj123
> <https://example.com/myApp/obj123>>. All you want to do is
> invoke one of its methods, to see what happens. Using the web_send
> library from the Firebug console, you could write:
>
> factory = lib.web.getLocation(); // grab the window.location URL
> drum = lib.Q.post(factory, 'makeDrum', []);
>
> That code generates the following HTTP request:
>
> POST /myApp/obj123?q=makeDrum HTTP/1.1
> Host: example.com
> Content-Type: text/plain; charset=UTF-8
> Content-Length: 2
>
> []
>
> The arguments to Q.post() are:
>
> 1. remote reference for the target object
> 2. optional argument to add to the query string
> 3. optional JSON value for the request body
>
> The makeDrum() method didn't take any arguments. Here's one that does:
>
> lib.Q.post(drum, 'bang', [ 1 ]);
>
> Causing the HTTP request:
>
> POST /myApp/obj456?q=bang HTTP/1.1
> Host: example.com
> Content-Type: text/plain; charset=UTF-8
> Content-Length: 5
>
> [ 1 ]
>
> The target URL in the above request was taken from the HTTP response
> to the previous request. For this to work, the web_send library
> introduces some conventions for HTTP requests and responses. These
> conventions are the least restrictive they can be, while still
> supporting the client-side remote reference API. This document
> explains these conventions and shows how to use legacy JSON resources
> that don't follow the conventions.
>
> ... (continue reading at <http://waterken.sourceforge.net/web_send/
> <http://waterken.sourceforge.net/web_send/>>)
>
> --
> "Waterken News: Capability security on the Web"
> http://waterken.sourceforge.net/recent.html
> <http://waterken.sourceforge.net/recent.html>
>
>

Gene Berger a écrit :
> You should only give your user's the options on a screen of what fields to
> choose, not have them write out the fields names themselves. (Is that what
> you are doing?)  You would face a big inject attack if you let them write
> the SQL fields themselves.

Yes the user would have a nice graphical interface to specify which
fields he wants. Also, I would validate the fields names on the backend
before creating the SQL query.

Cheers,
GFK's
--
Guillaume Filion
http://guillaume.filion.org/



[Non-text portions of this message have been removed]

On Thu, May 7, 2009 at 10:51 AM, Kris Zyp <kriszyp@...> wrote:
> This looks really cool.

Thanks!

>  Have you tried it with existing servers to see
> how compatible it is with current implementations of HTTP/JSON servers?
> This data interaction aspect of this looks similar to the functionality
> in Dojo's JsonRestStore, which can interact with CouchDB and Persevere.
> It'd be curious if your browser shell works with Persevere, as Persevere
> follows RFC 2616 pretty closely as well as some of the RESTful JSON
> conventions discussed here, such as JSON referencing:
> http://groups.google.com/group/restful-json

The implementation currently supports GET and POST to an arbitrary URL
with an arbitrary JSON request entity and an arbitrary JSON response
entity. Non-success HTTP response codes trigger the provided failure
callback.

Is there some aspect of Persevere's implementation of RFC 2616 that
could be problematic for a client implemented on the browser's
XMLHttpRequest object?

All JSON encoding/decoding is done using whatever JSON object is in
the global scope. It looks like you could use your JSON referencing
code by importing it instead of json2.js.

--Tyler

--
"Waterken News: Capability security on the Web"
http://waterken.sourceforge.net/recent.html