Using PUT for data entry causes fatal error
Just an update. Mango crashed again.
Can't work with this. Let me know what I can submit to sort this. We can't handle this crashing every few days.
Fox
EDIT: Looks like the setting of the ulimit values may not have taken hold as I had hooed... Now we wait...
@MattFox I am fairly confident in the source of your problem, but if the ulimit change does not fix it you can submit a zip file of all your log files to our support email and I will review them in detail. Also you probably have *.hprof files that are generated when the JVM crashes, submit the latest one of those as a zip as well.
Thanks Terry, I hope so too, the soft limit was at 1024 yet the hard limit is 1048576. This greatly confused me.
I've got 7800 points, though I do wonder if this means the cap gets hit really quick if I have 1000000 files open which are scattered across a few thousand points. I'm not overly familiar with the structure.of the NoSql DB.
After reviewing my settings I've managed to get the soft limit at 65535 and maintained the same hard limit. Part of me wonders if that limit should be higher...
Sorry to be difficult
Fox
Hi, Matt
I am just curious about your hardware configuration in this case. Is it a HTS or Mango running on your own Linux box?
Victor
Virtual dedicated cloud server,
To test to be sure I've used prlimit to set mango's file open limits just to be certain of limits. I'm finding for some stupid reason when I logged in before, ulimit maxes were back at 4096. Wish it'd make up its stupid mind... I may have to restart mango to make it use the new limits i've set in /etc/security/limits.conf but if this continues to work with no issue after setting the process limit perhaps I'll be ok...
Fox
Just crashed again, mango is still working in the background, got 11500 files open.
Web service crashed, cannot view dashboard, cannot access mango from browser,
API not talking either...
Shoot!
EDIT: Looks like the API keeps dropping in and out. CPU is high. Perhaps it's time to upgrade the machine's CPUs and memory.... Just managed to click back in to the dashboard...
EDIT 2: Have upgraded CPU to 6 cores now. Things appear to be running much smoother now. Will monitor and wait/see how things transpire now.
curious for 7800 points how often are they read from the datasources, how often are they logged to disk, and how much RAM are you allowing for the JVM on the 6 core CPU?
8g at the moment.
I've allowed for more processes to run so mango doesn't just sit still chewing ram.
I've yet to see it climb that high though. Higher CPU usage appears to be the culprit at the moment.
Just crashed again an hour ago.... -_-!
Cannot see any .hprof files anywhere...
I'm emailing logs now.
Fox
@MattFox I assume this was handled via support?
Correct, I'll post my fixes here once I've implemented them if that's helpful
