• phildunlap

    Good catches, Fox!

    Hi HSAcontrols, welcome to the forum!

    I think Fox may have set you on the right path for where the calculation may be going wrong. The only think I have to add is that the black question mark diamond is indeed a NaN value being rendered. I'd perhaps have to see a screenshot to be sure, but that's what it sounds like.

    I tested the script incrementally and only towards the end of the calculation does this error occur, maybe a problem with the division & square root being to small?

    NaN is a very infectious value. If one of the point values you're looping over is NaN, then doing any mathematical operation on it or with it will also produce NaN. You could check with an if(!isNaN(value_i)) {} around the calculations for DiffSquared and sumDiffSquared

    posted in Scripting general Discussion read more
  • phildunlap

    Thanks for taking a second look, reporting your experience, and the kind words!

    posted in User help read more
  • phildunlap

    cwangv

    that may well be the cause of your issue. Due to polling delay, the data will not come in at exact 15 minutes intervals.

    Not so! Mango is somewhat deceptive in terms or recorded timestamps in polls, which is why we discourage collecting highly time-precise data directly via Mango.

    Mango will schedule a poll for a data source at a certain time, and all data will report that timestamp for the poll, even if the poll took quite some time to run. That's not true for every type of data source, since some transmit the timestamps with the data, but for protocols like BACnet and Modbus all points sampled in the poll will have the poll as the timestamp of the value.

    The problem is somewhat more readily evident even with those sources now, since "Quantize" became an option on all data sources, which will align the period along the millisecond boundary of the polling period, so it is easy to collect data with these timestamps.

    cbyrne

    Hi @phildunlap, thanks for the info, however I think this way of calculating deltas is flawed for anyone doing metered readings. These happen at regular intervals like above and currently the delta value for an hour does not show the user their usage for one hour and this is the critical issue.

    I can see your argument. I will raise it with others and see their perspective. The reason the ending timestamp is exclusive is that otherwise values on the period boundary could be counted twice in separate periods, causing the sum, average, and integral to have issues when comparing to the whole period. But, as you are noting, the delta has that behavior.

    Ofsetting by 1 ms does not cure the problem as it forces the last reading in the period into the next period so it still gets missed.

    Offsetting the values by 1 ms backward would solve the display issue of those rollups, I would expect. Or one could offset the period 1 ms forward if not using the datebar and truncate is false.

    Similarly, would you consider adding a last_value-first value to the mango statistics object?

    There already are the values Start, First, and Last in terms of values. They are formally defined here: https://help.infiniteautomation.com/mango-rollups-and-statistics

    posted in User help read more
  • phildunlap

    To the stack trace involving,

    Caused by: org.h2.jdbc.JdbcSQLException: File corrupted while reading record: "index not found 93". Possible solution: use the recovery tool [90030-180]

    I would advise restoring a database backup. See this these resources for ways of doing that:
    https://help.infiniteautomation.com/how-to-restore-a-h2-database-backup
    https://forum.infiniteautomation.com/topic/2748/how-to-restore-a-database-backup

    To the stack trace involving,

    Caused by: java.net.BindException: Address already in use

    This usually means that Mango is already running on that port, but it could also mean that another application is running and has bound that port. The solution if Mango is already running is to access the Mango that is running, and if another application has bound it you can change the port Mango binds to in the Mango/overrides/properties/env.properties file.

    posted in MangoES Hardware read more
  • phildunlap

    This thread was my last go at simplest mass event detector creation, by leveraging watchlists: https://forum.infiniteautomation.com/topic/3708/a-script-jsonemport-example

    posted in User help read more
  • phildunlap

    I was not able to replicate. I downloaded over a hundred thousand values and still saw the second item way down in the response. What does the end of the JSON look like when it doesn't download both? Does it still close the JSON objects and lists properly?

    posted in User help read more
  • phildunlap

    Hi cbyrne,

    I would expect most of this is being caused by the precision of everything occurring exactly on the milliseconds of the 15 minute intervals. Mango statistics are computed with an inclusive period start time, but an exclusive period end time. So, with the fifteen minute data, or really any of the rollups you are trying that are multiples of that, can have this behavior in the delta. As far as the statistics are concerned, the 15 minutes started with a real value, then no other values occurred in the next 15 minutes, so the delta is zero.

    In the 30 minute example, there are two values in the period, so the delta can be computed, but because the period ends exactly when the next value is recorded, the delta between those two values is not accounted for in either period's delta.

    Try offsetting your periods by a millisecond - except be aware that doing rollups via the datebar will automatically align the data period to fit an integer number of rollup periods inside the data period by expanding start and end times to period boundaries. With REST requests this can be controlled via setting the truncate parameter false.

    posted in User help read more
  • phildunlap

    Hi JF89,

    Guessing on the red scratches (a screenshot would help) but that sounds to me like the component doesn't have an active point / has a disabled point associated with it.

    posted in Dashboard Designer & Custom AngularJS Pages read more
  • phildunlap

    My first request was very large and I still saw the second XID. If there is a limit on the payload in the response I don't think it is being imposed by Mango. In multiple arrays the limits apply to each series individually. You can request a 1 millisecond rollup of a series and download many GB all of the sudden (3.6 improved the memory footprint of huge data streams significantly). When you tried a limit of 1 did you always see both XIDs?

    posted in User help read more
  • phildunlap

    Just wondering how I should go about doing a backup through the command line in case my backup is a different version than the current interface. I've tried following a previous forum post but I'm not quite understanding the command line sequence.

    The post you linked to is about doing a restore on the command line. That command should be able to be run from your Mango directory, but you may need to change the path to the H2 JAR and the mah2 file, due to version numbers having changed. Do you have a more specific question?

    All of my data sources are there, however none of my points have their previous values.

    You can use the /ui/administration/no-sql page to "Merge Mango NoSQL data into Mango" by point it at the old_databases/mangoTSDB directory. Assuming you were using NoSQL, then your data would be in your old databases folder, presuming you moved it to be able to start fresh. The incremental backups are of the NoSQL data. You could possibly restore these to get your data back as well.

    posted in User help read more
  • phildunlap

    Just judging from the screenshot it looks like it would still pass the numeric value in the same format, so no it wouldn't simplify the bit-banging in a script part of things. It may simplify receiving the messages, though, since without parameter names the HTTP Receiver may have other issues to get past. It seems like that content type header is just wrong, it is not application/x-www-form-urlencoded data.

    posted in User help read more
  • phildunlap

    Do you have the ability to set the Content-Type header to application/json ? Otherwise you'll need to do something like catch the whole message with an HttpReceiver and parse the JSON / do the locating of points from within scripts. Only POSTs with content types containing application/json are passed to the JSON receiver data sources.

    Edit: This may not be so bad, if you'll need to do bit-banging to get the numeric value anyway (which it sounds like would be needed by that number encoding).

    It also took me a while to figure out the data format. Its a 32Bit float but each 8 bits is in decimal. I have gotten the correct values by converting each 8 bits to binary and inputting the result into a IEE754 converter. Can this be done with the data in mango?

    You can use the parseInt() function after you split the string of decimal values and pair it down. After that you'd need to bitshift and decode. Probably going to have to go through java.lang.Long (instantiate one with the lowest order byte, then bitshift and or the others into it) and java.lang.Double.longBitsToDouble() (pass this function the long from step 1)

    Maybe something like...

    //untested
    var message = JSON.parse(incoming.value);
    var regexValues = /.*(\d+),(\d+),(\d+),(\d+).*/.exec(message.VR);
    var longBits = new java.lang.Long(parseInt(regexValues[1]));
    longBits |= parseInt(regexValues[2]) <<  8;
    longBits |= parseInt(regexValues[3]) << 16;
    longBits |= parseInt(regexValues[4]) << 24;
    CONTEXT_POINTS[message.ID].set( java.lang.Double.longBitsToDouble( longBits ));

    posted in User help read more
  • phildunlap

    My notion of the hacky solution looked more like,

    <ma-get-point-value ng-init="latestTimeToggle=false; page={};" point-xid="brown" point="myPoint" on-value-updated="latestTimeToggle=!latestTimeToggle;"></ma-get-point-value>
    <ma-point-values ng-if="!latestTimeToggle" point="myPoint" values="page.point1Values" latest="1" realtime="false" use-cache="false"></ma-point-values>
    <ma-point-values ng-if="latestTimeToggle" point="myPoint" values="page.point1Values" latest="1" realtime="false" use-cache="false"></ma-point-values>
    <p>
        The point name is "{{myPoint.name}}" and its value is {{myPoint.renderedValue}} at {{myPoint.time}}
    </p>
    
    {{page.point1Values}}
    

    But probably the best solution, since you're already writing components / directives, would be to interact with the point value websocket to subscribe to the LOGGED event type, and then you could handle any messages of that type to update your component.

    See the /ui/docs/ng-mango-services/ma-point-event-manager page for how to use the point event manager service to subscribe to the event type in your component. You can see an example of its use in the ma-get-point-value directive, https://github.com/infiniteautomation/ma-dashboards/blob/main/UI/web-src/ngMango/directives/getPointValue.js#L76

    posted in User help read more
  • phildunlap

    I certainly did check. If you didn't have read permission to the second point it would come back with an "unauthorized" response, and would be a "not found" if no such point exists. Perhaps it would be more evident with less data returned. Did you try setting a limit, as with ?limit=1 added at the end of the URL?

    Edit: it could be there was corruption in the series, but then I would expect running the request again should work. Also it would be noted either in your Mango/logs/ma.log file or the Mango/logs/iastsdb-corruption.log

    posted in User help read more
  • phildunlap

    You should see the key for the second XID after all the values in the series for the first, as a key in the outermost object. This is what multiple arrays means, each point has its whole data series in an array off its XID, with no intermixing.

    posted in User help read more
  • phildunlap

    Hi Rob,

    Can you post what a JSON message from it would look like, and what you're trying to parse out of it?

    From what I see in your screenshot, it a point may look something like,
    0_1566392557769_somethinglikethis.png

    This thread may be helpful: https://forum.infiniteautomation.com/topic/4057/how-to-set-a-value-using-http/9

    posted in User help read more
  • phildunlap

    For the first example I think you'd have to write that as a new component in a user module.

    For the second, while there may be a CSS selector to get that done, the easiest way would be to copy Mango/classes/i18n.properties to Mango/overrides/properties/i18n.properties and changing

    common.relinquish=Relinquish to common.relinquish=Auto

    This may require restarting Mango, though. There may also be other instances of the word relinquish you may or may not want to change.

    posted in MangoES Hardware read more
  • phildunlap

    Glad we got it figured out!

    posted in User help read more
  • phildunlap

    Hi Ian,

    To me it sounds like if it were possible to have the <ma-get-point-value> tag subscribed to LOGGED events on the websocket instead of UPDATE would get most of what you're looking for, but I don't think that's an option.

    @Jared-Wiltshire may have a better solution, but I think you could use the on-value-updated function opportunity in the ma-get-point value to requery for the latest 1 from the ma-point-values directive. I can try to flesh that out for you but I'll give others time to chime in, since it seems there'd be a more direct way...

    posted in User help read more