Announcement

Collapse
No announcement yet.

nsJSON plug-in

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • daveb
    replied
    Just checking in again on these two problems in nsJSON....
    this is a great plugin but very difficult to use as we keep hitting the limit and getting truncated/invalid json packets as responses.
    These are the two problems detailed in my March 2022 post...

    #1: 256 char limit to a POST response
    #2: support for "RawOutput": false ...so that the "Output" node is proper json, not a string like this "{"key1":"value1", etc... }"

    Would be great if these two problems in nsJSON could be fixed... anyone? Or recommend a different json plugin if this one is deprecated?‚Äč

    Leave a comment:


  • Anders
    replied
    I tried contacting the author but no response so far.

    Leave a comment:


  • daveb
    replied
    Some progress to report and a workaround in my case.

    1. the truncation problem remains, and appears to be a limit of 256 characters in the POST response that appears in the "Output" node.

    2. Still getting empty ouput {} unless I set "RawOutput" to true. This gives the \" escaped quotes, but *provided* the string is not truncated (ie <256 characters) then "Output" is a wellformed string with a closing quote and can be loaded in then json nodes extracted from the response. Here's a summary of what I did... this can possibly be simplified but is definitely working within these limitations (mainly the 256 character response limit):


    nsJSON::Set /tree HttpWebRequest /value `{ "Url": "${HttpWebRequestURL}", "Verb": "POST", "DataType": "JSON", "RawOutput": true }`
    nsJSON::Set /tree HttpWebRequest "Headers" "Authorization" /value "Basic QQQQQetc"
    ;....etc for other headers...
    nsJSON::Set /tree HttpWebRequest "Data" "MyPayloadKey1" /value `"$R0"` ; $R0 has some payload data
    ;....etc for other payload keys, then do the POST....
    nsJSON::Set /tree HttpWebResponse /http HttpWebRequest

    ; check the response code
    nsJSON::Get /tree HttpWebResponse "StatusCode" /end
    Pop $R1
    StrCmp $R1 "200" +1 badstatus

    ; extract the "Output" node as POST response
    nsJSON::Get /tree HttpWebResponse "Output" /end
    Pop $R1
    nsJSON::Set /tree TheResponse /value `$R1`

    ; now we can look at the values of response keys like this
    nsJSON::Get /tree TheResponse "Message" /end ; this is the "Message" node in TheResponse
    Pop $R5
    DetailPrint `Here is TheResponse.Message node: [$R5]`

    nsJSON::Get /tree TheResponse "AnotherResponseKey" /end
    Pop $R6
    DetailPrint `Here is TheResponse.AnotherResponseKey node: [$R6]`

    ;....etc for other response nodes


    = = = = = =
    Note: I would have thought that providing a [Nodepath] as in the documentation would work, instead of first loading the "Output" node into another tree.
    Maybe that's due to dealing with the "RawOutput"? but this doesn't work...

    nsJSON::Get /tree HttpWebResponse "Output" "Message" /end
    Pop $R1

    ...giving an empty string in $R1. I'm guessing that the problem is that the actual "Output" node is a quoted and escaped string, not a json node even though it converts nicely to json when handed in to this line of code...

    nsJSON::Set /tree TheResponse /value `$R1`

    So that's a summary of my workaround for the moment.
    Would be great if these two problems in nsJSON could be fixed...
    #1: 256 char limit to a POST response
    #2: support for "RawOutput": false ...so that the "Output" node is proper json, not a string like this "{\"key1\":\"value1\", etc... }"

    Leave a comment:


  • daveb
    replied
    I also have been trying to hit a simple api with a json POST and get a json response.
    I experimented for quite a while and was always getting "Output":{} as reported by Minother.
    Testing was based on using Postman with the same json POST to the same server, and it was giving a proper json response.

    At last I tried setting "RawOutput": true
    This now gives me something closer... at least the Output node is now a string that resembles the correct/desired json response.
    Two catches...
    1. All the quotes (") in the Output node are now escaped as \"
    2. Even when I try to remove these \ characters, it isn't quite right.
    It appears to be truncating the first of what should be two subnodes, and the {} are now mismatched.

    Here's the (edited) sample output with the \" turned into " and some linefeeds added to make reading easier....

    {
    "Output": "{
    "License":{
    "SyncTime":"2022-03-25T00:43:34",
    "Status":"Active",
    "Expiry":"2022-05-02T00:00:00",
    "UserID":"900000",
    "Level":"-16641",
    "MaxVer":"3072",
    "AccNum":"XXXXX",
    "AccName":"YYYYYYYYYY YYYY",
    "AccEmail":"[email protected]",
    "PCFinger":"4380FB1YYYYYYY",
    "StatusCode": 200
    }

    Notice "Output": "{ etc... there is a spurious " before the opening {, and
    notice also, after "PCFinger":etc, it is just truncated and missing the rest of this node plus the whole of the next node.

    I really would like to read the response as json - it is valid json when I get it in a test using Postman. Certainly would be better to get the response as json, and be able to get subnodes etc. But as reported by Minother above, if I turn off RawOutput, I get an empty Output node with a correct "StatusCode": 200 in the response

    Any help appreciated.. I am new to using this nsJSON plugin which looks to be perfect for the job needed, if I could tame it...

    Leave a comment:


  • Anders
    replied
    I don't think storing a small JSON file on a SSD is going to hurt it much. Windows writes a lot more to the drive just in daily use.

    Leave a comment:


  • Minothor
    replied
    Originally Posted by Anders View Post
    From a quick look at the code, it looks like you can set "Agent" to whatever you want. Perhaps Github wants a "Mozilla" in there? From a quick test in another tool it looks like Github does not care much about the UA.

    Do you have access to a web server when you can put a json file for testing? I have never used this plug-in to read directly from the web so I don't know much about that part. As a last resort, use INetC to download to $pluginsdir and then parse that file.
    Sadly I do not have a webserver handy myself, I could set up a test on localhost in a pinch if needed.

    I'm leaning towards the INetC plugin too, since we already use it to download the binaries, but I'm unsure how to parse the body of a GET response as a string rather than persisting it on disk.
    (I'm refactoring the installer away from hardcoded values that have to be manually updated and trying to minimise needless disk writes since a lot of our users are likely to be running SSDs as their primary drive.)

    Thanks for bearing with me on this, it's been an interesting but long overdue first foray into NSIS.

    looks like it may be a similar issue to the ones Ahmett linked in Post 64.
    Swapping to that version gave me partial output, but it looks like the response is getting cut off part way through the "assets_url" node of the reply.

    I'll keep digging.

    Leave a comment:


  • Anders
    replied
    From a quick look at the code, it looks like you can set "Agent" to whatever you want. Perhaps Github wants a "Mozilla" in there? From a quick test in another tool it looks like Github does not care much about the UA.

    Do you have access to a web server when you can put a json file for testing? I have never used this plug-in to read directly from the web so I don't know much about that part. As a last resort, use INetC to download to $pluginsdir and then parse that file.

    Leave a comment:


  • Minothor
    replied
    Originally Posted by Anders View Post
    This seems wrong to me. What is `Output`? I think you need to break this down into smaller steps and confirm each little step is working. Focus on getting a correct value into $arraySize before moving on.
    Hi, thanks for responding so quickly, the reason for "Output" being part of the path was due to that first segment of code I linked*, it's the JSON tree associated with the Async response.
    It appeared as though the tree had the body of the response under a node named "Output", so I prepended that to the path and tested again.

    As it currently stands, both Async or regular HTTP GET requests to that URL seem to produce a 200 (OK) response but an empty body.
    (even when serializing the tree immediately upon completion of the request)

    So far, I haven't been able to replicate that behaviour in a browser or using the Rested Firefox plugin to replicate the User-Agent and headers that nsJSON is sending - both the browser and the plugin return the expected information from GitHub's API.

    At this point I can only assume it's either some bug or I've overlooked something particularly silly, hence coming here to ask persons more knowledgeable and familiar with both NSIS and this plugin.

    *A big thanks to whoever edited my post to fix the code blocks by the way, I couldn't find an Edit option available to me.

    Leave a comment:


  • Anders
    replied
    nsJSON::Get /tree asyncHttpWebResponse /count `Output` `assets` /end
    This seems wrong to me. What is `Output`? I think you need to break this down into smaller steps and confirm each little step is working. Focus on getting a correct value into $arraySize before moving on.

    Leave a comment:


  • Minothor
    replied
    Could someone point me in the right direction on this?

    I'm trying to load JSON from GitHub's API, but the response I'm getting in the output file is:

    PHP Code:
    {
        
    "Output": {
            
        },
        
    "StatusCode"200

    Comparing the Empty output to the expected result of an unauthenticated GET request: https://api.github.com/repos/HoboVR-...eleases/latest

    Leaves me confused as to where exactly I'm going wrong.

    I've read and re-read the ReadMe and HttpWebRequest.nsi example file, I've checked my var, tree names and paths and like the Response body - I'm drawing a blank.


    Test Script:
    PHP Code:
    !include nsDialogs.nsh
    !include MUI2.nsh
    !include "winmessages.nsh"
    !include LogicLib.nsh
    !include "Locate.nsh"
    !include registry.nsh
    !include x64.nsh

    ;!define LATEST_RUN "https://api.github.com/repos/HoboVR-Labs/hobo_vr/actions/runs?per_page=1&branch=master&event=push&status=success"
    ;"hobovr-build-windows-latest"

    Name "HoboVR"

    !define httpWebRequestURL "https://api.github.com/repos/HoboVR-Labs/hobo_vr/releases/latest"

    !insertmacro MUI_LANGUAGE English

    Var windowsBinaryURL
    Var currentIndex
    Var arraySize
    Var nodeContentType

    Section
    ; [url]https://api.github.com/repos/HoboVR-Labs/hobo_vr/releases/latest[/url]
    ; -> assets -> [COUNT] -> iterate:
    "application/x-ms-dos-executable"

      
    ;Get the latest successful Run

        nsJSON
    ::Set /tree httpWebRequest /value `{ "Url": "${httpWebRequestURL}", "Verb": "GET", "Async": true , "Decoding": true , "AccessType": "PreConfig" }`
        
    DetailPrint `Getting latest build from: ${httpWebRequestURL}`
      ;  
    DetailPrint `Download: ${HttpWebRequestURL}`
        
    nsJSON::Set /tree asyncHttpWebResponse /http httpWebRequest

        
    Wait until done.
        ${Do}
          
    Sleep 1000
          nsJSON
    ::Wait httpWebRequest /timeout 0
          Pop $R0
          
    ${If} $R0 != wait
            DetailPrint 
    `Request complete!`
            ${Break}
          ${EndIf}
          
    DetailPrint `Waiting...`
        ${
    Loop}

      ; 
    Get the Artifacts array
      
    nsJSON::Get /tree asyncHttpWebResponse /count `Output` `assets` /end
      Pop $arraySize

      StrCpy $currentIndex 0

      IntCmp $arraySize 0 artifactNotFound artifactNotFound

      DetailPrint 
    "$arraySize compiled binaries found..."

      
    checkNextArtifact:
      
    DetailPrint `Checking Artifact at Index: $currentIndex...`
      
    nsJSON::Get /tree asyncHttpWebResponse `Output` `assets` /index $currentIndex `content_type` /end
      Pop $nodeContentType

      DetailPrint 
    "Artifact ContentType: $nodeContentType"
      
    StrCmp $nodeContentType "application/x-ms-dos-executable" artifactFound

      IntOp $currentIndex $currentIndex 
    1
      IntCmp $currentIndex $arraySize artifactNotFound checkNextArtifact

      artifactFound
    :
      
    nsJSON::Get /tree asyncHttpWebResponse `Output` `assets` /index $currentIndex `browser_download_url` /end
      Pop $windowsBinaryURL
      DetailPrint 
    "Found binary: $windowsBinaryURL"
      
    Goto end

      artifactNotFound
    :
      
    DetailPrint `No Binary could be found.`
      
    CreateDirectory $EXEDIR\Output
      nsJSON
    ::Serialize /tree asyncHttpWebResponse /format /file $EXEDIR\Output\asyncHttpWebResponse.json
      DetailPrint 
    `Response saved to: $EXEDIR\Output\AsyncHttpWebResponse.json`

      
    end:
      
    DetailPrint `Test Finished.`

    SectionEnd 

    Leave a comment:


  • T.Slappy
    replied
    @Afrow: In the most recent version there is a small bug:

    If there is an empty array like this:

    {
    "ChunkDbs": [],
    "FormatVersion": 0
    }
    The Json (read from file in my case) is neved read correctly and the Error flag is set, however empty arrays are perfectly correct in Json.


    Edit:

    Quick fix is to update the function with another if condition:

    static struct JSON_NODE* EatNode(PTCHAR pszBuffer, int* piPos, BOOL bIsValue)

    PHP Code:
    ...
            else if (
    EatChar(pszBufferpiPosTEXT('[')))
            {
                
    pNode->eType JNT_ARRAY;
                if (
    EatChar(pszBufferpiPosTEXT(']'))) // <<<< Start: This fixes empty arrays
                
    {
                    
    pNode->eType JNT_VALUE;
                    
    pNode->pszValue TEXT("null");
                }                             
    // <<<< End fix
                
    else
                {
                    
    pNode->pValue EatNodeArray(pszBufferpiPos);
                    if (!
    pNode->pValue || !EatChar(pszBufferpiPosTEXT(']')))
                    {
                        
    JSON_Delete(&pNodeNULL);
                            return 
    NULL;
                    }
                }
            }
    ... 
    Last edited by T.Slappy; 29 January 2021, 09:25. Reason: Added fixed code

    Leave a comment:


  • ahmett
    replied
    some more detail for an ANSI setup;

    NsJSON v1.1.0.6 is right version for these;
    nsJSON::Set /file /Unicode (Don't work in v1.1.1.0)
    nsJSON::Serialize /format /file /Unicode (Don't work both in v1.1.1.0 and 1.1.0.9)

    Leave a comment:


  • ahmett
    replied
    It fails to read Unicode JSON files in ANSI setup! (/Unicode tag only works in Unicode setup builds)



    I've now checked the previous version and it does not have this problem!

    Leave a comment:


  • Anders
    replied
    Can't handle empty objects?

    ${IfNotThen} ${FileExists} "$%TEMP%\test.txt" ${|} ExecWait `"$sysdir\cmd.exe" /C echo.{"node1":{node122:{x:0}}} > "$%TEMP%\test.txt"` ${|}
    ClearErrors
    nsJSON::Set /file "$%TEMP%\test.txt"
    ${IfThen} ${Errors} ${|} MessageBox mb_ok Err ${|}
    nsJSON:elete 'node1' 'node122' /end
    nsJSON::Set 'node1' 'node122' /value '{deleteme:0}'
    nsJSON:elete 'node1' 'node122' 'deleteme' /end
    nsJSON::Set node3 node31 node313 node3131 a31313 /value "Z1Z1Z1"
    nsJSON::Serialize /format /file "$%TEMP%\test.txt"
    This works the first time, displays error if you run it again because node122 is {}?

    Leave a comment:


  • r2du-soft
    replied
    nsJSON get value

    hi
    i read a json value from variable but i have some problems:

    this code can't show result value address:
    HTML Code:
    StrCpy $0 '{"status":"ok","username":"rose","password":"software","expire":"2020","used_time":[],"address":"http:\/\/google.com\/apps"}'
    
    
    
    nsJSON::Set /value $0
    
    nsJSON::Get "address" /end
    Pop $R0
    DetailPrint $R0

    but this code show result value address:
    HTML Code:
    StrCpy $0 '{"status":"ok","username":"rose","password":"software","expire":"2020","used_time":"[]","address":"http:\/\/google.com\/apps"}'
    
    
    
    nsJSON::Set /value $0
    
    nsJSON::Get "address" /end
    Pop $R0
    DetailPrint $R0


    HTML Code:
    The difference in codes is [] and is "[]"
    my problem is the [] , if the value is entered in double quotation "[]" the result successfully displayed
    but if in variable not insert [] to quotation then result can't successfully displayed

    Leave a comment:

Working...
X
😀
🥰
🤢
😎
😡
👍
👎