nsJSON plug-in

[Afrow UK]

A JSON (JavaScript Object Notation) parser, manipulator and generator plug-in for NSIS. See the readme file for usage and more information.

http://nsis.sf.net/File:NsJSON.zip

Stu

[T.Slappy]

Nice work!
Please write a short article into wiki (or copy from readme): http://nsis.sourceforge.net/NsJSON_plug-in

[Yathosho]

very nice, especially since none of the xml plugins worked to my satisfaction and i got the option to use json files for the same purpose. thanks stu!

[Brummelchen]

can i use it to parse json backup files from firefox (bookmark backup file/places)?

[Afrow UK]

If it is valid JSON then yes.

Stu

[Afrow UK]

New version:

1.0.0.1 - 1st July 2012
* Fixed parsing of single digit numbers.
* Fixed Serialize not writing the output file when the stack isn't empty.

I've tested this version on Chrome's Preferences JSON file.

Stu

[Afrow UK]

New version:

1.0.0.2 - 15th August 2012
* Fixed Unicode build parsing and serializing.

Note the Unicode build still expects ASCII encoded JSON files at the moment. I will perhaps add support for UTF-8 encoded JSON in the future.

Stu

[T.Slappy]

Hey Stu, is it possible to parse floating numbers with comma instead of point?
Some currencies use it: "1.23" is "1,23"

[Afrow UK]

You have to wrap the value in quotes so that it will be treated as a string rather than a float. It is not possible any other way as a comma is used as the delimiter between objects and values (whitespace is ignored).

Stu

[Afrow UK]

New version (with a fair number of changes):

1.0.1.0 - 28th August 2012
* Added /unicode switch to the Serialize function. Output files for both plug-in builds are now encoded in ANSI by default.
* Removed the Parse function in favour of Set /file [/unicode].
* Added /type, /key, /exists, /count, /isempty to the Get function.
* Added /index switch for referencing nodes by index.

http://nsis.sourceforge.net/File:NsJSON.zip

Stu

[Marshallx7]

Awesome plugin Stu. Any chance of extending it to support multiple json instances open at once?

[Afrow UK]

That is already supported kind of. You just need to use Set /file with a node path to load a JSON file into. The only thing you cannot do is serialize a specific node to a file, which is where a change would be necessary. How much do you need multiple instance support (give an example of usage)?

Stu

[Marshallx7]

I'm using in-memory json to store lots of my own user-specific data rather than tons of global variables.
Then I want to open up a program's settings file which is also in json, add a value and resave that settings.json.

It's also useful to separate different blocks of data and only serialize the parts I need during debug.

I guess the only thing that's really needed is the ability to serialize a specific node.

Unless I can find a better solution for my in-memory storage... Actually, thinking about it I could probably use your arrays plugin and do arrays of pointers to arrays using a counter or concatenating names like
CarPark0 = array of license plates (e.g. 0=ABC123, 1=DEF456, 2=GHI789)
CarPark0_ABC123 = array of that particular car's properties (e.g. Make=ShinyThingsInc, Color=Green)

So yeah, it probably isn't necessary to have the json plugin handle more than 1 root json object at a time. I'll just write my own serializer for the arrays.

Cheers again for your awesome stuff.

[zeeh3]

I am getting truncated string values when they have spaces:

nsJSON::Set `more_stuff` `PSCHED_009_NAME` /value `QoS Packet Scheduler`

Results in:

"more_stuff": { "PSCHED_009_NAME": QoS }

[Afrow UK]

Values are parsed as JSON. If you want quoted strings you must use double quotes around the value:

nsJSON::Set `more_stuff` `PSCHED_009_NAME` /value `"QoS Packet Scheduler"`

Stu

[maliktous]

nsJSON::Serialize crashes nsis installer with recent stable chrome default Preferences, which contain a very long (~66 KBytes) `variations_seed` string parameter.

Test case:
Fresh install of google chrome (33.0.1750.154) which contain `variations_seed` parameter.

code:
nsJSON::Set /file "$Prefs"

nsJSON::Serialize /format /file "$Prefs"

I'm using ANSI NSIS v2.46 with nsJSON 1.0.1.0.

Preferences sample:
http://pastebin.com/raw.php?i=Xs4HLBMN

deleting `variations_seed` before serializing workarounds this, but is ugly

code:
nsJSON :: Delete `variations_seed` /end

[maliktous]

PHP Code:

nsJSON::Set /file "$Prefs"
nsJSON::Serialize /format /file "$Prefs" 


with following json:

PHP Code:

{
   "selectfile": {
      "last_directory": "E:\\"
   },
   "session": {
      "restore_on_startup_migrated": true,
      "startup_urls_migration_time": "13037468179710507"
   }
} 


saves broken json:

PHP Code:

{
    "selectfile": {
        "last_directory": "E:\\"
   },
   "
    }
} 


[Yathosho]

Originally Posted by maliktous

PHP Code:

{
   "selectfile": {
      "last_directory": "E:\\"
   },
} 


It take it the plugin escapes the last quote, but wouldn't you have to escape the backslash as well?

[Anders]

Originally Posted by Yathosho

It take it the plugin escapes the last quote, but wouldn't you have to escape the backslash as well?

"E:\\" should end up as "E:\", there is no escaped quote because that double backslash was already used to create a backslash

[maliktous]

Originally Posted by Yathosho

It take it the plugin escapes the last quote, but wouldn't you have to escape the backslash as well?

1. It is a standart Google Chrome Preferences, so it is a valid json. I did not create it manually. I just read it, add my values, then save it.
2. Python parses it correctly:

PHP Code:

$ cat test.json
{
   "selectfile": {
      "last_directory": "E:\\"
   },
   "session": {
      "restore_on_startup_migrated": true,
      "startup_urls_migration_time": "13037468179710507"
   }
}
>>> f=open("test.json")
>>> import json
>>> json.load(f)
{u'selectfile': {u'last_directory': u'E:\\'}, u'session': {u'restore_on_startup_migrated': True, u'startup_urls_migration_time': u'13037468179710507'}} 


[Yathosho]

Originally Posted by Anders

"E:\\" should end up as "E:\", there is no escaped quote because that double backslash was already used to create a backslash

ah, of course. you're right

[maliktous]

Originally Posted by maliktous

nsJSON::Set /file "$Prefs"
nsJSON::Serialize /format /file "$Prefs"
with following json:
saves broken json:

workaround (should be used before Set /file ):

PHP Code:

Push `\\"`          # text to be replaced
Push `"`            # replace with
Push all            # replace all occurrences
Push all            # replace all occurrences
Push ${PrefsFile}   # file to replace in
Call AdvReplaceInFile 


[Afrow UK]

I will try to look at the issues this weekend.

Stu

[Afrow UK]

1.0.1.1 - 29th March 2014
* Fixed incorrect handling of escape character (\).

Originally Posted by maliktous

nsJSON::Serialize crashes nsis installer with recent stable chrome default Preferences, which contain a very long (~66 KBytes) `variations_seed` string parameter.

I could not reproduce this with the new build so it may be fixed. The full output string (66224 bytes) is correctly written to the output file for both the ANSI and Unicode versions. No memory access violations were caught in the debugger either.

Stu

[CherryDT]

I can confirm that there is still a problem. The problem happens when the data is >64KB. It doesn't always crash, though, because it's corrupting the heap which might or might not result in a crash.

The source of the problem is that in the Serialize function, you allocate 65536 bytes for pszArg:

code:
PTCHAR pszArg = (PTCHAR)LocalAlloc(LPTR, sizeof(TCHAR) * 65536);

...and later you are using this buffer to store the serialized string:

code:
bOK = JSON_Serialize(g_root, pszArg, bIsFile, bAsUnicode, bFormat);

There is no length check taking place anywhere.

[Afrow UK]

I will work on a fix to automatically increase the buffer size this weekend.

Stu

[Afrow UK]

New version:

1.0.1.2 - 12th July 2014
* Fixed crash on serialization to file for node values larger than 64KB.
* Fixed crash on serialization to stack for JSON larger than NSIS_MAX_STRLEN. The JSON will now be truncated.

The file serialization doesn't use any temporary buffer any more (it just writes directly to the file) so no reallocation was necessary.

Stu

[maliktous]

1.0.1.2 crashes if JSON file starts with Byte order mark

It seems that `nsJSON::Set /file $JSON` works but the next `nsJSON::Set` crashes installer.

I don't know if json can contain BOM, so it is just for your attention, but I guess, at least, it should not crash.

--- "Bad" 2014-09-05 15:53:16.000000000 +0300
+++ "Good" 2014-09-05 15:53:16.000000000 +0300
@@ -1,4 +1,4 @@
-<U+FEFF>{
+{
"test": {

[Afrow UK]

New version:

1.0.1.3 - 18th October 2014
* Added UTF-16LE BOM, UTF-16BE BOM and UTF-8 signature detection for input files (with UTF-16BE conversion to UTF-16LE).
* Fixed formatting errors for the Serialize function.
* Fixed closing bracket or curly brace not being included on Serialize to stack when not using /format.

This should resolve the crash for files with UTF-8 signatures.

My next task is to add HTTP GET/PUT/POST support for use with REST.

Stu

[MeTaNoV]

Hi,
I am trying to use nsJSON plug-in and just find out that comment '//' does disturb somehow the parser, and break the reading of any node inside the file...
I'll temporally avoid using comment until I find a proper solution!
Cheers,
Pascal

[Anders]

JSON does not really have comments AFAIK.

The safest option is to use _mycomment:"comment text" style entries.

If you still want a real comment then /**/ block comments probably have a higher chance of working across parsers...

[Marshallx7]

This plugin is awesome.

However, is there any chance of adding an escape function to convert an NSIS string into a JSON string?

For example, right now if I call
nsjson::Set some node path /value ":"
Then the colon is not stored because it is invalid JSON, not a string. I have to call
nsjson::Set some node path /value `":"`

But if I don't know where the text has come from e.g.
nsjson::Set some node path /value $R0
Then that's a problem.

One of the following would be ideal:
nsjson::escape $R0
Pop $R0
nsjson::Set some node path /value $R0

alternatively,
nsjson::Set some node path /value /escape $R0

Such a function shouldn't try to determine the type of data - just treat the input as an unescaped, unquoted string.

[JasonFriday13]

Try quotes or double quotes around the variable? Like: "$R0". If strings have spaces, they can resolve to multiple arguments when used with a plugin, which is why quotes are needed.

[Afrow UK]

1.1.0.1 - 23rd November 2015
* Added Quote function.
* Added amd64-unicode build.

1.1.0.0 - 19th April 2015
* Support for multiple JSON trees.
* JSON via HTTP web requests.

Somehow I hadn't uploaded the last version I'd been working on so I apologise for that. It includes built-in support for multiple JSON trees and HTTP web requests (e.g. for REST APIs).

http://nsis.sourceforge.net/File:NsJSON.zip

Stu

[Marshallx7]

This is great, thanks Stu

[Marshallx7]

NB: When you use nsJSON::Get /type, if the nodepath doesn't exist then nothing gets pushed onto the stack - so always make sure you check that the nodepath /exists before you try to get the type.

Stu, you could change /type to return empty string if the node path doesn't exist, and do away with /exists altogether.

[Marshallx7]

Hey, Stu
The new Quote function doesn't escape strings whose first and last characters are already quotemarks. For example, nsjson::Quote `"some"text"`

[Afrow UK]

Originally Posted by Marshallx7

Hey, Stu
The new Quote function doesn't escape strings whose first and last characters are already quotemarks. For example, nsjson::Quote `"some"text"`

The output is:

"some\"text"

This is correct assuming the original feature request was to only surround the string in quotes if it isn't so already. Nevertheless:

1.1.0.2 - 9th December 2015
* Get function /type switch returns an empty string if the node does not exist.
* Added /always switch to Quote function which surrounds the input string in quotes even if it is already quoted.

Therefore, use the /always switch and the output will instead be:

"\"some\"text\""

http://nsis.sourceforge.net/File:NsJSON.zip

Stu

[Marshallx7]

This is really great Stu, thanks.

Readme needs updating with latest switches behavior btw.

I have another feature request if you feel inclined:

{ a: 'a', b: 1, c: {}}
nsJSON::Get /keys ...
Returns on the stack:
3
a
b
c

[Yathosho]

Originally Posted by Marshallx7

{ a: 'a', b: 1, c: {}}

Erm, I would expect the plugin to deny this input, since it's invalid JSON