nsJSON plug-in

[Marshallx7]

That was just an example of the JSON object in memory (as valid Javascript rather than JSON), not the string passed to nsJSON to build it. The point was the /keys feature request.

[Marshallx7]

I have also found another case where nsJSON::Get does not return anything on the stack:
If you try to get something that doesn't exist (e.g. the specified /tree does not exist)

Please can this be changed to return empty string, to avoid stack corruption?

And I think that nsJSON:elete causes NSIS to crash if you try to delete something that doesn't exist.

[Afrow UK]

The plug-in will set the error flag if the tree or node does not exist. If you think it is possible the tree or node may not exist, you should check the error flag before reading from the stack. Remember to clear the error flag first if necessary. The error flag is the only way to distinguish a definite error.

I have implemented /keys and fixed the Delete crash but I am working on a new feature for console application execution for JSON input/output so will upload a new version when done.

Stu

[Marshallx7]

Awesome, thanks Stu!

[Marshallx7]

Just discovered another crash, not sure if this is related to the previous crash in Delete:

nsJSON::Set /tree testTree testNode /value testValue
nsJSON::Get /tree testTree /exists /end
Pop $R0
MessageBox MB_OK $R0 ; "yes"
nsJSON::Delete /tree testTree /end ; CRASH

[Marshallx7]

I'm also having trouble using the unicode version of the plugin:

code:
OutFile "test.exe"

!define SETTING_UNICODE

!ifdef SETTING_UNICODE

    Unicode true

    !addplugindir /x86-unicode "./plugins/x86-unicode"

!else

    Unicode false

    !addplugindir /x86-ansi "./plugins/x86-ansi"

!endif

Page instfiles

Function .onInit

	nsjson::set /tree testTree testNode /value testValue

    nsjson::get /tree testTree testNode /end

    Pop $R0

    MessageBox MB_OK $R0 ; "testValue"

FunctionEnd

Section "dummy"

SectionEnd

With the ansi version, the message box reads "testValue" as expected, but with the unicode version is just says "t".

[JasonFriday13]

Originally Posted by Marshallx7

I'm also having trouble using the unicode version of the plugin:

code:
OutFile "test.exe"

!define SETTING_UNICODE

!ifdef SETTING_UNICODE

    Unicode true

    !addplugindir /x86-unicode "./plugins/x86-unicode"

!else

    Unicode false

    !addplugindir /x86-ansi "./plugins/x86-ansi"

!endif

Page instfiles

Function .onInit

	nsjson::set /tree testTree testNode /value testValue

    nsjson::get /tree testTree testNode /end

    Pop $R0

    MessageBox MB_OK $R0 ; "testValue"

FunctionEnd

Section "dummy"

SectionEnd

With the ansi version, the message box reads "testValue" as expected, but with the unicode version is just says "t".

Sounds like you are using the ansi version for both ansi and unicode installers.

By the way, if the plugins are copied correctly to both '${NSISDIR}\plugins\x86-ansi' and '${NSISDIR}\plugins\x86-unicode', then you don't need to use !addplugindir. 'Unicode True' does that work for you.

[Marshallx7]

Nope, confirmed I am using the correct builds. DLLs are stored in local plugins directory, not in NSISDIR, hence the use of addplugindir. Does this work for you?

[JasonFriday13]

No it doesn't work for me, it's showing the same as what you are getting.

[Anders]

When something in a ANSI installer is unexpectedly a single character then it is a clear indication that something was not converted from Unicode to ANSI. The other way around could be anything, a plugin mismatch or bug. A missing conversion bug to Unicode usually looks Chinese.

[Afrow UK]

1.1.0.3 - 4th March 2016
* Fixed crash for the Delete function when specifying a tree/path that
does not exist.
* Fixed Unicode build.
* Added /keys switch to Get function.

http://nsis.sf.net/File:NsJSON.zip

I am still working on the process exec code.

Stu

[Marshallx7]

Yep, Unicode works fine now, thanks Stu.

I have another feature request:
It would be great if /index accepted negative numbers to get/set/delete from the end of an array/node.
For example, /index -1 means the last element, /index -2 means the second-last element and so on.

Right now, this is achieved by getting the count of elements, decrementing, then getting/setting/deleting the required element. A negative index would streamline this process.

[Afrow UK]

1.1.0.5 - 10th August 2016
* Empty value keys ("") were not included in serialized JSON.
* Serialize function popped one extra value from the stack.

1.1.0.4 - 7th June 2016
* Fixed incorrect flags used for HTTP requests.
* Added support for negative indices.
* Added console application execution for input and output.
* Added support for asynchronous HTTP requests and console application
execution.

http://nsis.sf.net/File:NsJSON.zip

Stu

[awall]

Hi Stu,
started using nsJSON recently, very nice work!

I'm facing an issue using HTTP web requests:

code:
nsJSON::Set /tree Request /value `{ "Url": "${myURI}", "Verb": "POST", "DataType": "JSON" }

nsJSON::Set /tree Response /http Request

The REST method I'm calling in the "Url" may have an empty response (e.g. only returns 200 OK).
In such case, the returned Response tree has the "Output" node populated with the response from the previous call to another REST method, instead of returning an empty "Output" node.

Is this a bug or am I missing something?

Thanks
Andrea

[Afrow UK]

I have uploaded a new version which fixes the issue.

http://nsis.sourceforge.net/File:NsJSON.zip

Stu

[Marshallx7]

I'm having a problem with the latest build where nsJSON::Serialize is leaving the filepath on the stack. I know there was an issue fixed recently where one too many values were popped but now the reverse seems to be true...

I suspect that the old code was always popping file, even if it wasn't needed, whereas the new code pushes it back on the stack even if it was needed

On a separate note, I have found that doing an http request with Verb=POST and DataType=JSON sends a request where the post data is unicode with Content-Type: application/x-www-form-urlencoded
Is there a way to get this to send as UTF-8 with Content-Type: application/json ?

Thanks

[Anders]

How are you supposed to delete a tree? Especially the "default"? tree?

nsJSON::Set /value '{ "Input": [ { "test1": false, } ] }'
nsJSON::Delete /end # <-- Crash on exit!

nsJSON::Set /value '{ "Input": [ { "test1": false, } ] }'
nsJSON::Delete "" /end # <-- Does nothing

nsJSON::Set /value '{ "Input": [ { "test1": false, } ] }'
nsJSON::Set /value '{}' # <-- "Works"

nsJSON::Set /value '[]'
nsJSON::Set /value '{ "Input": [ { "test1": false, } ]' # Not valid JSON now?
nsJSON::Set /value '{}' # <-- Now it fails

nsJSON::Delete /tree "" /end # <-- Crash

[Victor4]

Good job!
Try this tool: JSON formatter

[Afrow UK]

Originally Posted by Anders

How are you supposed to delete a tree? Especially the "default"? tree?

1.1.1.0 - 21st November 2017
* Fixed JSON with syntax errors still being parsed without setting the error flag.
* Fixed Set function not replacing the root value if the value was an array.
* Fixed Delete function not deleting the root node and tree.

http://nsis.sourceforge.net/File:NsJSON.zip

Anders, please let me know if this resolves all your issues. Apologies for the wait.

This will delete the root node and tree:

code:
nsJSON::Delete /end

Stu

[Marshallx7]

I'm having a problem with the latest version (1.1.1.0)

code:
nsjson::Set x /value []

nsjson::Get /exists x /end

Pop $R0

MessageBox MB_OK "$R0" ;yields "no", expected "yes"



nsjson::Set x /value [1]

nsjson::Get /exists x /end

Pop $R0

MessageBox MB_OK "$R0" ;yields "yes", as expected

To be clear, the previous version of the plugin (1.1.0.9) gives "yes" for the first snippet, as expected.

[r2du-soft]

hi
i read a json value from variable but i have some problems:

this code can't show result value address:

StrCpy $0 '{"status":"ok","username":"rose","password":"software","expire":"2020","used_time":[],"address":"http:\/\/google.com\/apps"}'



nsJSON::Set /value $0

nsJSON::Get "address" /end
Pop $R0
DetailPrint $R0

but this code show result value address:

StrCpy $0 '{"status":"ok","username":"rose","password":"software","expire":"2020","used_time":"[]","address":"http:\/\/google.com\/apps"}'



nsJSON::Set /value $0

nsJSON::Get "address" /end
Pop $R0
DetailPrint $R0

The difference in codes is [] and is "[]"

my problem is the [] , if the value is entered in double quotation "[]" the result successfully displayed
but if in variable not insert [] to quotation then result can't successfully displayed

[Anders]

Can't handle empty objects?

${IfNotThen} ${FileExists} "$%TEMP%\test.txt" ${|} ExecWait `"$sysdir\cmd.exe" /C echo.{"node1":{node122:{x:0}}} > "$%TEMP%\test.txt"` ${|}
ClearErrors
nsJSON::Set /file "$%TEMP%\test.txt"
${IfThen} ${Errors} ${|} MessageBox mb_ok Err ${|}
nsJSON::Delete 'node1' 'node122' /end
nsJSON::Set 'node1' 'node122' /value '{deleteme:0}'
nsJSON::Delete 'node1' 'node122' 'deleteme' /end
nsJSON::Set node3 node31 node313 node3131 a31313 /value "Z1Z1Z1"
nsJSON::Serialize /format /file "$%TEMP%\test.txt"

This works the first time, displays error if you run it again because node122 is {}?

[ahmett]

It fails to read Unicode JSON files in ANSI setup! (/Unicode tag only works in Unicode setup builds)

https://nsis.sourceforge.io/File:NsJSON.zip

I've now checked the previous version and it does not have this problem!

[ahmett]

some more detail for an ANSI setup;

NsJSON v1.1.0.6 is right version for these;
nsJSON::Set /file /Unicode (Don't work in v1.1.1.0)
nsJSON::Serialize /format /file /Unicode (Don't work both in v1.1.1.0 and 1.1.0.9)

[T.Slappy]

@Afrow: In the most recent version there is a small bug:

If there is an empty array like this:

{
"ChunkDbs": [],
"FormatVersion": 0
}

The Json (read from file in my case) is neved read correctly and the Error flag is set, however empty arrays are perfectly correct in Json.


Edit:

Quick fix is to update the function with another if condition:

static struct JSON_NODE* EatNode(PTCHAR pszBuffer, int* piPos, BOOL bIsValue)

PHP Code:

...
        else if (EatChar(pszBuffer, piPos, TEXT('[')))
        {
            pNode->eType = JNT_ARRAY;
            if (EatChar(pszBuffer, piPos, TEXT(']'))) // <<<< Start: This fixes empty arrays
            {
                pNode->eType = JNT_VALUE;
                pNode->pszValue = TEXT("null");
            }                             // <<<< End fix
            else
            {
                pNode->pValue = EatNodeArray(pszBuffer, piPos);
                if (!pNode->pValue || !EatChar(pszBuffer, piPos, TEXT(']')))
                {
                    JSON_Delete(&pNode, NULL);
                        return NULL;
                }
            }
        }
... 


[Minothor]

Could someone point me in the right direction on this?

I'm trying to load JSON from GitHub's API, but the response I'm getting in the output file is:

PHP Code:

{
    "Output": {
        
    },
    "StatusCode": 200
} 


Comparing the Empty output to the expected result of an unauthenticated GET request: https://api.github.com/repos/HoboVR-...eleases/latest

Leaves me confused as to where exactly I'm going wrong.

I've read and re-read the ReadMe and HttpWebRequest.nsi example file, I've checked my var, tree names and paths and like the Response body - I'm drawing a blank.


Test Script:

PHP Code:

!include nsDialogs.nsh
!include MUI2.nsh
!include "winmessages.nsh"
!include LogicLib.nsh
!include "Locate.nsh"
!include registry.nsh
!include x64.nsh

;!define LATEST_RUN "https://api.github.com/repos/HoboVR-Labs/hobo_vr/actions/runs?per_page=1&branch=master&event=push&status=success"
;"hobovr-build-windows-latest"

Name "HoboVR"

!define httpWebRequestURL "https://api.github.com/repos/HoboVR-Labs/hobo_vr/releases/latest"

!insertmacro MUI_LANGUAGE English

Var windowsBinaryURL
Var currentIndex
Var arraySize
Var nodeContentType

Section
; [url]https://api.github.com/repos/HoboVR-Labs/hobo_vr/releases/latest[/url]
; -> assets -> [COUNT] -> iterate:
; "application/x-ms-dos-executable"

  ;Get the latest successful Run

    nsJSON::Set /tree httpWebRequest /value `{ "Url": "${httpWebRequestURL}", "Verb": "GET", "Async": true , "Decoding": true , "AccessType": "PreConfig" }`
    DetailPrint `Getting latest build from: ${httpWebRequestURL}`
  ;  DetailPrint `Download: ${HttpWebRequestURL}`
    nsJSON::Set /tree asyncHttpWebResponse /http httpWebRequest

    ; Wait until done.
    ${Do}
      Sleep 1000
      nsJSON::Wait httpWebRequest /timeout 0
      Pop $R0
      ${If} $R0 != wait
        DetailPrint `Request complete!`
        ${Break}
      ${EndIf}
      DetailPrint `Waiting...`
    ${Loop}

  ; Get the Artifacts array
  nsJSON::Get /tree asyncHttpWebResponse /count `Output` `assets` /end
  Pop $arraySize

  StrCpy $currentIndex 0

  IntCmp $arraySize 0 artifactNotFound artifactNotFound

  DetailPrint "$arraySize compiled binaries found..."

  checkNextArtifact:
  DetailPrint `Checking Artifact at Index: $currentIndex...`
  nsJSON::Get /tree asyncHttpWebResponse `Output` `assets` /index $currentIndex `content_type` /end
  Pop $nodeContentType

  DetailPrint "Artifact ContentType: $nodeContentType"
  StrCmp $nodeContentType "application/x-ms-dos-executable" artifactFound

  IntOp $currentIndex $currentIndex + 1
  IntCmp $currentIndex $arraySize artifactNotFound checkNextArtifact

  artifactFound:
  nsJSON::Get /tree asyncHttpWebResponse `Output` `assets` /index $currentIndex `browser_download_url` /end
  Pop $windowsBinaryURL
  DetailPrint "Found binary: $windowsBinaryURL"
  Goto end

  artifactNotFound:
  DetailPrint `No Binary could be found.`
  CreateDirectory $EXEDIR\Output
  nsJSON::Serialize /tree asyncHttpWebResponse /format /file $EXEDIR\Output\asyncHttpWebResponse.json
  DetailPrint `Response saved to: $EXEDIR\Output\AsyncHttpWebResponse.json`

  end:
  DetailPrint `Test Finished.`

SectionEnd 


[Anders]

nsJSON::Get /tree asyncHttpWebResponse /count `Output` `assets` /end

This seems wrong to me. What is `Output`? I think you need to break this down into smaller steps and confirm each little step is working. Focus on getting a correct value into $arraySize before moving on.

[Minothor]

Originally Posted by Anders

This seems wrong to me. What is `Output`? I think you need to break this down into smaller steps and confirm each little step is working. Focus on getting a correct value into $arraySize before moving on.

Hi, thanks for responding so quickly, the reason for "Output" being part of the path was due to that first segment of code I linked*, it's the JSON tree associated with the Async response.
It appeared as though the tree had the body of the response under a node named "Output", so I prepended that to the path and tested again.

As it currently stands, both Async or regular HTTP GET requests to that URL seem to produce a 200 (OK) response but an empty body.
(even when serializing the tree immediately upon completion of the request)

So far, I haven't been able to replicate that behaviour in a browser or using the Rested Firefox plugin to replicate the User-Agent and headers that nsJSON is sending - both the browser and the plugin return the expected information from GitHub's API.

At this point I can only assume it's either some bug or I've overlooked something particularly silly, hence coming here to ask persons more knowledgeable and familiar with both NSIS and this plugin.

*A big thanks to whoever edited my post to fix the code blocks by the way, I couldn't find an Edit option available to me.

[Anders]

From a quick look at the code, it looks like you can set "Agent" to whatever you want. Perhaps Github wants a "Mozilla" in there? From a quick test in another tool it looks like Github does not care much about the UA.

Do you have access to a web server when you can put a json file for testing? I have never used this plug-in to read directly from the web so I don't know much about that part. As a last resort, use INetC to download to $pluginsdir and then parse that file.

[Minothor]

Originally Posted by Anders

From a quick look at the code, it looks like you can set "Agent" to whatever you want. Perhaps Github wants a "Mozilla" in there? From a quick test in another tool it looks like Github does not care much about the UA.

Do you have access to a web server when you can put a json file for testing? I have never used this plug-in to read directly from the web so I don't know much about that part. As a last resort, use INetC to download to $pluginsdir and then parse that file.

Sadly I do not have a webserver handy myself, I could set up a test on localhost in a pinch if needed.

I'm leaning towards the INetC plugin too, since we already use it to download the binaries, but I'm unsure how to parse the body of a GET response as a string rather than persisting it on disk.
(I'm refactoring the installer away from hardcoded values that have to be manually updated and trying to minimise needless disk writes since a lot of our users are likely to be running SSDs as their primary drive.)

Thanks for bearing with me on this, it's been an interesting but long overdue first foray into NSIS.

looks like it may be a similar issue to the ones Ahmett linked in Post 64.
Swapping to that version gave me partial output, but it looks like the response is getting cut off part way through the "assets_url" node of the reply.

I'll keep digging.

[Anders]

I don't think storing a small JSON file on a SSD is going to hurt it much. Windows writes a lot more to the drive just in daily use.

[daveb]

I also have been trying to hit a simple api with a json POST and get a json response.
I experimented for quite a while and was always getting "Output":{} as reported by Minother.
Testing was based on using Postman with the same json POST to the same server, and it was giving a proper json response.

At last I tried setting "RawOutput": true
This now gives me something closer... at least the Output node is now a string that resembles the correct/desired json response.
Two catches...
1. All the quotes (") in the Output node are now escaped as \"
2. Even when I try to remove these \ characters, it isn't quite right.
It appears to be truncating the first of what should be two subnodes, and the {} are now mismatched.

Here's the (edited) sample output with the \" turned into " and some linefeeds added to make reading easier....

{
"Output": "{
"License":{
"SyncTime":"2022-03-25T00:43:34",
"Status":"Active",
"Expiry":"2022-05-02T00:00:00",
"UserID":"900000",
"Level":"-16641",
"MaxVer":"3072",
"AccNum":"XXXXX",
"AccName":"YYYYYYYYYY YYYY",
"AccEmail":"daveb@zzzzzzzzzz.com",
"PCFinger":"4380FB1YYYYYYY",
"StatusCode": 200
}

Notice "Output": "{ etc... there is a spurious " before the opening {, and
notice also, after "PCFinger":etc, it is just truncated and missing the rest of this node plus the whole of the next node.

I really would like to read the response as json - it is valid json when I get it in a test using Postman. Certainly would be better to get the response as json, and be able to get subnodes etc. But as reported by Minother above, if I turn off RawOutput, I get an empty Output node with a correct "StatusCode": 200 in the response

Any help appreciated.. I am new to using this nsJSON plugin which looks to be perfect for the job needed, if I could tame it...

[daveb]

Some progress to report and a workaround in my case.

1. the truncation problem remains, and appears to be a limit of 256 characters in the POST response that appears in the "Output" node.

2. Still getting empty ouput {} unless I set "RawOutput" to true. This gives the \" escaped quotes, but *provided* the string is not truncated (ie <256 characters) then "Output" is a wellformed string with a closing quote and can be loaded in then json nodes extracted from the response. Here's a summary of what I did... this can possibly be simplified but is definitely working within these limitations (mainly the 256 character response limit):


nsJSON::Set /tree HttpWebRequest /value `{ "Url": "${HttpWebRequestURL}", "Verb": "POST", "DataType": "JSON", "RawOutput": true }`
nsJSON::Set /tree HttpWebRequest "Headers" "Authorization" /value "Basic QQQQQetc"
;....etc for other headers...
nsJSON::Set /tree HttpWebRequest "Data" "MyPayloadKey1" /value `"$R0"` ; $R0 has some payload data
;....etc for other payload keys, then do the POST....
nsJSON::Set /tree HttpWebResponse /http HttpWebRequest

; check the response code
nsJSON::Get /tree HttpWebResponse "StatusCode" /end
Pop $R1
StrCmp $R1 "200" +1 badstatus

; extract the "Output" node as POST response
nsJSON::Get /tree HttpWebResponse "Output" /end
Pop $R1
nsJSON::Set /tree TheResponse /value `$R1`

; now we can look at the values of response keys like this
nsJSON::Get /tree TheResponse "Message" /end ; this is the "Message" node in TheResponse
Pop $R5
DetailPrint `Here is TheResponse.Message node: [$R5]`

nsJSON::Get /tree TheResponse "AnotherResponseKey" /end
Pop $R6
DetailPrint `Here is TheResponse.AnotherResponseKey node: [$R6]`

;....etc for other response nodes


= = = = = =
Note: I would have thought that providing a [Nodepath] as in the documentation would work, instead of first loading the "Output" node into another tree.
Maybe that's due to dealing with the "RawOutput"? but this doesn't work...

nsJSON::Get /tree HttpWebResponse "Output" "Message" /end
Pop $R1

...giving an empty string in $R1. I'm guessing that the problem is that the actual "Output" node is a quoted and escaped string, not a json node even though it converts nicely to json when handed in to this line of code...

nsJSON::Set /tree TheResponse /value `$R1`

So that's a summary of my workaround for the moment.
Would be great if these two problems in nsJSON could be fixed...
#1: 256 char limit to a POST response
#2: support for "RawOutput": false ...so that the "Output" node is proper json, not a string like this "{\"key1\":\"value1\", etc... }"

[Anders]

I tried contacting the author but no response so far.