-
MattFox
Hi Jared, just giving this a bump and wondering if you were able to replicate this issue
Fox
-
MattFox
I hope they're helpful! Looking back it looks a little bit like gibberish! Gimme a bell if you need anything
Fox
-
MattFox
I've still got the API dropping in and out, can't always log in. Can I get someone to remote in and take a look?
Fox
-
MattFox
Good idea with the thread dumps, if I can get through to have them exported during an import session I will post them here.
Thanks for your input here. Perhaps some feedback regarding my server configuration may help here so I can document best practices for running and getting the best out of the mango system.
Seeing how if SSL with http2 is the way to go perhaps I need to rejig how my server is configured.
Let's see how I goFox
-
MattFox
I know the system drops out when we are updating the values via the mango API every 5 minutes.
I'm looking at/v1/point-values Update one or many data point's current value
But am unsure if this will still apply an update event to affected datapoints, that and I'm not sure of the correct format.
thought it might be something like{ xid:"aasdf", timestamp:12314556677, value:10, type:"NUMERIC" }
I'll try the max open files later tonight if you feel that could be a bottle neck if I have several 100 different points coming in with a 100 values at a time.
-
MattFox
Thanks for coming back to me at this ungodly hour Terry, sorry I'm stressing a little bit here due to being outside my knowledge base...
I'm still on 3.5.6, I don't want to do any upgrades due to the time of year and that our customers are more active during the spring/summer/autumn seasons and rely on our systems being up. Trying to maintain 99.9% uptime is a gripper.. That and I'm afraid of any breaking changes created could really throw a spanner in the development works...I confirmed it's not the proxy as I loaded mango's pages directly and sure enough the pages dropped out.
# Copyright (C) 2014 Infinite Automation Systems Inc. All rights reserved. # @author Matthew Lohbihler ############################################################################### # TO OVERRIDE VALUES IN THIS FILE... # # Do not change the values in this file, because when you upgrade your core # your changes will be overwritten. Instead, create a new file called # <MA_HOME>/overrides/properties/env.properties and override properties # there. The overrides directory will never be overwritten by an upgrade, so # your customizations will be safe. # ############################################################################### # The port at which Mango Automation will listen for browser connections web.port=8080 # The host interface to which Mango Automation will bind and listen for new connections # 0.0.0.0 is the special interface that will force a bind to all available interfaces web.host=0.0.0.0 # Should Mango Automation open (if possible) a browser window when it starts up? web.openBrowserOnStartup=true # Web caching settings # disable caching web.cache.noStore=false web.cache.noStore.rest=true web.cache.noStore.resources=false # set max age of cached files in seconds, only if noStore=false # versioned resources are those with ?v=xxx on the query string web.cache.maxAge=0 web.cache.maxAge.rest=0 web.cache.maxAge.resources=86400 web.cache.maxAge.versionedResources=31536000 #Upload file size limit (bytes) -1 means no limit web.fileUpload.maxSize=50000000 # Default database settings, NOTE that on windows this must be an absolute path #db.type=h2 #db.url=jdbc:h2:${ma.home}/databases/mah2 #db.location=${ma.home}/databases/mah2 #db.username= #db.password= #For web console #db.web.start=false #db.web.port=8091 #to compact the database size at shutdown (may take longer but will free up disk space) #db.h2.shutdownCompact=false #db.type=derby #db.url=${ma.home}/databases/madb #db.username= #db.password= #General Database Settings db.pool.maxActive=100 db.pool.maxIdle=10 db.update.log.dir=${ma.home}/logs/ # setting to show query times in the logs as INFO db.useMetrics=false # if set, will only log slow queries, above this threshold in ms. Will be logged at WARN level instead of INFO db.metricsThreshold=100 #--The following database properties are for RQL REST queries and can be changed during runtime and will be picked up at most in 5s-- #Force the use of indexes (Experimental and only on MySQL so far) db.forceUseIndex=true #Tell the jdbc driver to fetch this many rows at a time, useful over network connected dbs (Not MySQL) # negative values will force use jdbc driver default db.fetchSize=-1 #Tell the database to not return the entire result set (or fetch blocks) but to return row by row # can slow down performance on network systems but reduce memory footprint for large queries db.stream=false #-- End auto-reloading Database Properties -- # MySQL database settings. Your MySQL instance must already be running and configured before this can be used. #db.type=mysql #db.url=jdbc:mysql://localhost/<your mysql schema name> #db.username=<your mysql username> #db.password=<your mysql password> #db.mysqldump=<location/command for mysqldump executable for backups> #db.mysql=<location/command for mysql executable for restore> db.mysqldump=<location/command for mysqldump executable for backups> db.mysql=<location/command for mysql executable for restore> # MySQL database settings. Your MySQL instance must already be running and configured before this can be used. db.type=mysql db.url=jdbc:mysql://[DB_IP]/mangodb db.username={USER} db.password={PASS} db.mysqldump=/usr/bin/mysqldump db.mysql=/usr/bin/mysql # Database settings for conversion. If the db.* settings point to a new database instance, and the convert type setting # is set, Mango Automation will attempt to convert from the convert.db.* settings to the db.* settings # Note that database conversions should not be performed in the same step as an upgrade. First upgrade, then convert. #convert.db.type=h2 #convert.db.url=jdbc:h2:/mnt/disks/sdb/mango/databases/mah2 #convert.db.username= #convert.db.password= #convert.db.type= #convert.db.url=${convert.db.url} #convert.db.username=${convert.db.username} #convert.db.password=${convert.db.password} #Set the base path for where the NoSQL data will be stored #db.nosql.location=${ma.home}/databases/ db.nosql.location=/mnt/disks/sdb/mango/databases/ #Set the folder name of the point value store db.nosql.pointValueStoreName=mangoTSDB #Set the number of files the database can have open at one time db.nosql.maxOpenFiles=500 #Time after which a shard will be closed db.nosql.shardStalePeriod=36000000 #Period to check for stale shards db.nosql.flushInterval=300000 #Query Performance Tuning, File Access Type: Available[INPUT_STREAM,FILE_CHANNEL,RANDOM_ACCESS_FILE,MAPPED_BYTE_BUFFER] db.nosql.shardStreamType=MAPPED_BYTE_BUFFER #Setting to speed up NoSQL queries at the expense of a small increase in disk usage db.nosql.reversible=true #Setting this will convert your existing point value store [NONE, REVERSIBLE, UNREVERSIBLE] db.nosql.convert=NONE #Number of concurrent threads to use to convert the database db.nosql.convertThreads=4 #Run the corruption scan if the db is marked dirty db.nosql.runCorruptionOnStartupIfDirty=false #Password encryption scheme [BCRYPT, SHA-1, NONE] #Legacy is SHA-1, 2.8+ BCRYPT #security.hashAlgorithm=BCRYPT # The location of the Mango Automation store from which to get license files. store.url=https://store.infiniteautomation.com # SSL control # *** NOTE *** # You can generate a self-signed certificate for testing using the following command # keytool -genkey -keyalg RSA -alias mango -keystore /location/to/keystore/file.jks -validity 365 -keysize 2048 # Enter keystore password: {type your keystore password <ENTER>} # Re-enter new password: {type your keystore password <ENTER>} # What is your first and last name? # [Unknown]: {the hostname mango is running on e.g. mymangotest.com OR localhost <ENTER>} # What is the name of your organizational unit? # [Unknown]: {e.g. Mango testing <ENTER>} # What is the name of your organization? # [Unknown]: {e.g. Infinite Automation Systems Inc. <ENTER>} # What is the name of your City or Locality? # [Unknown]: {e.g. Erie <ENTER>} # What is the name of your State or Province? # [Unknown]: {e.g. Colorado <ENTER>} # What is the two-letter country code for this unit? # [Unknown]: {e.g. US <ENTER>} # Is CN=localhost, OU=Development, O=Infinite Automation Systems Inc., L=Erie, ST=Colorado, C=US correct? # [no]: {type yes <ENTER>} # # Enter key password for <mango> # (RETURN if same as keystore password): {type your key password or just press <ENTER>} # Note: Enabling SSL also turns on HSTS which may not be desirable, see below ssl.on=false ssl.port=8443 ssl.keystore.location=/location/to/keystore/file.jks ssl.keystore.password=freetextpassword # If they key password is commented out, it is assumed to be the same as the keystore password #ssl.key.password= #Time socket can be idle before being closed (ms) ssl.socketIdleTimeout=70000 #Enable ALPN (Application-Layer Protocol Negotiation) for HTTP/2 # on current browsers HTTP/2 is only available for TLS/SSL connections. # Note that with this setting you must also have the ALPN script extension enabled for Mango to start. # (Adds -javaagent:${MA_HOME}/boot/jetty-alpn-agent-x.x.x.jar to the java options) ssl.alpn.on=true #Show debug output for alpn connections in log ssl.alpn.debug=false # Configure HSTS (HTTP Strict Transport Security) # Enabled by default when ssl.on=true # Sets the Strict-Transport-Security header, web browsers will always connect using HTTPS when they # see this header and they will cache the result for max-age seconds ssl.hsts.enabled=true ssl.hsts.maxAge=31536000 ssl.hsts.includeSubDomains=false # System time zone. Leave blank to use default VM time zone. timezone= #Rest API Configuration rest.enabled=true #Enable to make JSON More readable rest.indentJSON=true #Cross Origin Request Handling rest.cors.enabled=true rest.cors.allowedOrigins=* rest.cors.allowedMethods=PUT,POST,GET,OPTIONS,DELETE rest.cors.allowedHeaders=content-type,x-requested-with,authorization rest.cors.exposedHeaders= rest.cors.allowCredentials=true rest.cors.maxAge=3600 #For rest API Documentation at /swagger-ui.html swagger.enabled=true #path to api-docs for swagger tools springfox.documentation.swagger.v2.path=/swagger/v2/api-docs #By default the api-docs endpoint used by the swagger-ui is protected # if it is allowed to be public then one can generate tokens in for Users # to supply to the swagger-ui and access the api endpoints # To use, enter: Bearer <space> <token value> into the Authorize value input in the swagger ui swagger.apidocs.protected=true #Regex Pattern to scan for REST API endpoints for Swagger to document/display swagger.mangoApiVersion=v[12] #Distributor Settings distributor=IA #Jetty Thread Pool Tuning # Time a thread must be idle before killing to keep pool size at minimum web.threads.msIdleTimeout=10000 # Number of threads to keep around to handle incoming connections web.threads.minimum=10 # Number of threads allowed to be created to handle incoming requests as needed web.threads.maximum=250 # Number of Requests To Queue if all threads are busy web.requests.queueSize=200 # Ping timeout for response from browser web.websocket.pingTimeoutMs=20000 #Time socket can be idle before being closed (ms) web.socketIdleTimeout=30000 #Jetty JSP Configuration # See here for options: http://www.eclipse.org/jetty/documentation/9.2.10.v20150310/configuring-jsp.html web.jsp.development=true web.jsp.genStringsAsCharArray=true web.jsp.trimSpaces=false web.jsp.classdebuginfo=false web.jsp.supressSmap=true web.jsp.compilerClassName=org.apache.jasper.compiler.JDTCompiler web.jsp.compiler=modern web.jsp.fork=false web.jsp.keepgenerated=true #iFrame Header Control iFrame Header Control 'X-Frame-Options' (case sensitive options) # SAMEORIGIN - Only allow Mango to embed i-frames when the requesting page was loaded from the Mango domain # DENY - Do not allow at all # ANY - Do not even use the header at all # One specific domain name can be supplied so that the header becomes: ALLOW-FROM http://foo.bar.com web.security.iFrameAccess=SAMEORIGIN #Regex used to match serial ports so they show up in the menu serial.port.linux.regex=((cu|ttyS|ttyUSB|ttyACM|ttyAMA|rfcomm|ttyO|COM)[0-9]{1,3}|rs(232|485)-[0-9]) serial.port.linux.path=/dev/ serial.port.windows.regex= serial.port.windows.path= serial.port.osx.path=/dev/ serial.port.osx.regex=(cu|tty)..* #Start data sources in parallel threads runtime.datasource.startupThreads=8 #Log startup times for runtime manager runtime.datasource.logStartupMetrics=true #Log number of aborted polls for a polling data source this often at a minimum (only logged after next aborted poll past this time) runtime.datasource.pollAbortedLogFrequency=3600000 #Report Javascript Execution Times at INFO Level logging # add this to log4j.xml <category name="org.perf4j.TimingLogger"><level value="info"/></category> runtime.javascript.metrics=false #Default task queue size for the Real Time Timer, should multiple tasks of the same type be queued up? # Tasks are rejected from a full queue, a size of 0 means reject multiple instances of the same task runtime.realTimeTimer.defaultTaskQueueSize=0 #When a task queue is full should the waiting tasks be discarded and replaced with the most recent runtime.realTimeTimer.flushTaskQueueOnReject=false #Delay (in ms) to wait to rate limit task rejection log messages so they don't fill up logs and use too much cpu doing it runtime.taskRejectionLogPeriod=10000 #Maximum counts to wait to terminate the thread pool's tasks that are running or queued to run # each count is 1 second. So the default of 60 = 1 minute. Note that the medium and low # timeout happens first and then the remaining time is spent waiting of the high priority tasks. # So by setting both to the same value will result in waiting only as long as that value. runtime.shutdown.medLowTimeout=60 runtime.shutdown.highTimeout=60 # Set the location of the file stores # If not set, the location is MA_HOME/filestore filestore.location= # Limits the rate at which an unauthenticated IP address can access the REST API Defaults to an initial 10 request burst then 2 requests per 1 second thereafter rateLimit.rest.anonymous.enabled=true rateLimit.rest.anonymous.burstQuantity=10 rateLimit.rest.anonymous.quanitity=2 rateLimit.rest.anonymous.period=1 rateLimit.rest.anonymous.periodUnit=SECONDS # Limits the rate at which an authenticated user can access the REST API Disabled by default rateLimit.rest.user.enabled=false rateLimit.rest.user.burstQuantity=20 rateLimit.rest.user.quanitity=10 rateLimit.rest.user.period=1 rateLimit.rest.user.periodUnit=SECONDS # Limits the rate at which authentication attempts can occur by an IP address Defaults to an initial 5 attempt burst then 1 attempt per 1 minute thereafter rateLimit.authentication.ip.enabled=true rateLimit.authentication.ip.burstQuantity=5 rateLimit.authentication.ip.quanitity=1 rateLimit.authentication.ip.period=1 rateLimit.authentication.ip.periodUnit=MINUTES # Limits the rate at which authentication attempts can occur against a username Defaults to an initial 5 attempt burst then 1 attempt per 1 minute thereafter rateLimit.authentication.user.enabled=true rateLimit.authentication.user.burstQuantity=5 rateLimit.authentication.user.quanitity=1 rateLimit.authentication.user.period=1 rateLimit.authentication.user.periodUnit=MINUTES Follow symbolic links when serving files from Jetty web.security.followSymlinks=true # Content Security Policy settings, please see https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP # The reasons for the default policy are outlined below # style-src 'unsafe-inline' - inline styles are used by AngularJS Material for the dynamic theming # script-src 'unsafe-eval' - needed by Fabric.js used in amCharts for drawing on charts, also gives AngularJS a 30% performance boost # connect-src ws: wss: - necessary as 'self' does not permit connections to websockets on the same origin, this should be configured to restrict it to your server's actual hostname # img-src data: - allows for small base64 encoded images to be embedded inline into the html # img-src/script-src https://www.google-analytics.com - allows for enabling Google analytics (not enabled by default, must be manually enabled by admin via UI Settings page) # img-src/script-src https://maps.google.com https://maps.googleapis.com https://maps.gstatic.com - allows for using the Google maps component # style-src/font-src https://fonts.googleapis.com https://fonts.gstatic.com - allows for using Google fonts in dashboards web.security.contentSecurityPolicy.enabled=false web.security.contentSecurityPolicy.reportOnly=false web.security.contentSecurityPolicy.defaultSrc='self' web.security.contentSecurityPolicy.scriptSrc='self' 'unsafe-eval' https://maps.google.com https://maps.googleapis.com https://www.google-analytics.com web.security.contentSecurityPolicy.styleSrc='self' 'unsafe-inline' https://fonts.googleapis.com web.security.contentSecurityPolicy.connectSrc='self' ws: wss: web.security.contentSecurityPolicy.imgSrc='self' data: https://maps.google.com https://maps.gstatic.com https://www.google-analytics.com web.security.contentSecurityPolicy.fontSrc='self' https://fonts.gstatic.com web.security.contentSecurityPolicy.mediaSrc= web.security.contentSecurityPolicy.objectSrc= web.security.contentSecurityPolicy.frameSrc= web.security.contentSecurityPolicy.workerSrc= web.security.contentSecurityPolicy.manifestSrc= web.security.contentSecurityPolicy.other= # script-src 'unsafe-inline' - inline scripts are used extensively throughout the Mango legacy UI # script-src 'unsafe-eval' - The Dojo JS library uses eval() # style-src 'unsafe-inline' - inline styles are used throughout the Mango legacy UI # connect-src ws: wss: - necessary as 'self' does not permit connections to websockets on the same origin, this should be configured to restrict it to your server's actual hostname # img-src data: - allows for small base64 encoded images to be embedded inline into the html # img-src/script-src https://www.google-analytics.com - allows for enabling Google analytics web.security.contentSecurityPolicy.legacyUi.enabled=false web.security.contentSecurityPolicy.legacyUi.reportOnly=false web.security.contentSecurityPolicy.legacyUi.defaultSrc='self' web.security.contentSecurityPolicy.legacyUi.scriptSrc='self' 'unsafe-inline' 'unsafe-eval' https://www.google-analytics.com web.security.contentSecurityPolicy.legacyUi.styleSrc='self' 'unsafe-inline' web.security.contentSecurityPolicy.legacyUi.connectSrc='self' ws: wss: web.security.contentSecurityPolicy.legacyUi.imgSrc='self' data: https://www.google-analytics.com web.security.contentSecurityPolicy.legacyUi.fontSrc= web.security.contentSecurityPolicy.legacyUi.mediaSrc= web.security.contentSecurityPolicy.legacyUi.objectSrc= web.security.contentSecurityPolicy.legacyUi.frameSrc= web.security.contentSecurityPolicy.legacyUi.workerSrc= web.security.contentSecurityPolicy.legacyUi.manifestSrc= web.security.contentSecurityPolicy.legacyUi.other= # Set the location of the modules data directory, (relative to $MA_HOME if not absolute) If not set, the location is $MA_HOME/data moduleData.location=data # HTTP session (authentication) cookie name and domain name settings. # # Use the Mango GUID as the session cookie name sessionCookie.useGuid=true # name takes precedence over useGuid if set sessionCookie.name= # Set the domain name that the cookie is valid for, can be used to make the session login valid for subdomains too. If left blank the session cookie can only be used for the domain that you login at. sessionCookie.domain=
Let me know if there's anything I can do here to increase performance
Fox
-
MattFox
I keep seeing the mangoAPI server coming offline and online, to the point of interrupting user logins. What are your suggestions to stop this? I've got customers complaining about proxy errors (502) due to mango's webservice timing out.
Any suggestions would be greatFox
-
MattFox
I'm consistently getting this popping up after sending data created by a meta data point to be sent to a http server for parsing and forwarding.
Would this be happening now by the sheer number of points I'm firing out over time?
I've got 233 total going out by this publisher atm...
Am I getting these errors because they're not going through? They do return to normal after a while but I'm running low on brain power atm so I'm unable to deduce quickly what the issues could be.. -
MattFox
Correct, I'll post my fixes here once I've implemented them if that's helpful
-
MattFox
Sorry to storm in but I think that'd be beneficial for a lot of people Jared. Also thanks for your input, I think I've lost sight of a few things lately!
-
MattFox
I'm a little confused as to why you'd want to go that far but I believe you'd do something along the lines of..
/opt/mango36/overrides/web/modules/mangoUI/web/dev/controllers/pushCtrl.jsdefine(['angular', 'require'], function (angular, require) { pushCtrl.$inject = ['$scope','$timeout']; function pushCtrl($scope,$timeout) { const $ctrl = this; $ctrl.$onChanges = function (changes) { if (changes.points && Array.isArray($ctrl.points)) { $ctrl.point = $ctrl.points.find(p => p.xid === 'DP_ind_' + $ctrl.target); $ctrl.switchPoint = $ctrl.points.find(p => p.xid === 'DP_sw_' + $ctrl.target); $ctrl.ptCSS = { // point css values for color and blink; ex, ng-class="" get color(){ if ($ctrl.point.color < 10) { return 0; } return Math.floor($ctrl.point.value / 10)}, get blink(){return 'blink' + $ctrl.point.value % 10}, get colStyle(){return 'color'+this.color} }; $timeout(function() { console.log('color:', $ctrl.ptCSS.colStyle, 'blink:', $ctrl.ptCSS.blink ); },1000); } }; $ctrl.pushed = function() { console.log($ctrl.switchPoint.value); if ($ctrl.switchPoint) { $ctrl.switchPoint.toggleValue(); console.log('switch ' + $ctrl.target + ' was switched.' ); } }; } return pushCtrl; )};//define
then pushLamp.js:
define(['angular', 'require','/modules/web/mangoUI/web/dev/controllers/pushCtrl.js'], function (angular, require,pushCtrl) { return { templateUrl: '/modules/mangoUI/web/dev/push/push.html', bindings: { points: '<', target: '@' }, controller: pushCtrl }; });//define
I'm confident you can leverage require in the userModule.js to shorten that controller call into a single word but I've not done a great deal of angularJS lately due to other work commitments....
Fox
-
MattFox
I believe that stuff is pre set in the browser cookies. Have a look at that with the maUiApp stuff it remembers the configuration for that there.
There won't be any docs per se, you'll simply have to go through all the source and read how it is all pieced together. -
MattFox
Fixed, just mixed up the full path of the file with the userModule URL after closer inspection.
-
MattFox
I know what i've done wrong....
I didn't return the user module!
So sorry man!define(['angular', 'require','./components/pushLamp.js'], function (angular, require, pushLamp) { var userModule = angular.module('userModule', ['maUiApp']); userModule.filter('unsafe', ['$sce', function($sce) { return $sce.trustAsHtml; }]); try { userModule.component('pushLamp',pushLamp); } catch(e) { console.log('error: ', e); } return userModule; //MISSING THIS!!! aerghhh I hate mondays... });
Man I need a swift kick...
-
MattFox
ps aux | grep java
This will tell us what mango runs as. Also can you also print
ls -la /opt/mango36/overrides/web/modules/mangoUI/web/dev
-
MattFox
nope that's definitely correct, check your directory permissions and the owner.
I'll wager mango itself may not have access to the file.
Whatever the user runs as in mango set that as your modules etc directories for a start.
up to web. everything inside that shouldn't matter.We'll start with that and work our way up from there...
sudo chown [user]:[user] /opt/mango36/overrides -R
sudo find /opt/mango36/overrides -type d -exec chmod 755 {} \;
-
MattFox
if you're in linux, can you navigate to the directory where your usermodule is and enter
pwd
otherwise if in windows, open a command prompt in the location and enter
DIR
to print the path
-
MattFox
nope, just wanted to make sure you did have the files inside the mango's installation directory regardless of how it is named.
Did the http requests from your browser work?
If the mango is on localhost it'd be localhost/modules/mangoUI/web/dev/userModule.js
-
MattFox
Hang on, your mango install is in
/opt/mango36
?
Not in/opt/mango
? -
MattFox
You've got the right idea, the first thing I suggest is to see if you can even manually load the userModule from your web browser
[mangoURL]/modules/mangoUI/web/dev/userModule.js
if you can pull that, then try with the components directory