TIBCO EMS – No memory for operation – Filesystem as datastore.
Very often we come across this kind of an error :-
Most probably if your EMS is running as a 32bit Binary and your message size is exceeding 1.6 – 2 GB in size
EMS For 32 bit has a size limitation of 2 GB
When you are using Filesystem as a datastore,
and when you come across such a trace
2007-07-24 10:46:38 Recovering state, please wait.
2007-07-24 10:48:47 SEVERE ERROR: Failed writing message to ‘datastore/meta.db’: No memory for operation.
2007-07-24 10:48:50 SEVERE ERROR: No memory processing purge record.
2007-07-24 10:48:52 FATAL: Exception in startup, exiting.
434692644 Jul 24 11:16 meta.db
29220 Jul 24 08:54 sync-msgs.db
5805854244 Jul 24 08:37 async-msgs.db
Don’t Panic, Its a very Generic issue
Just follow these checklists
- Size of your message
- Ensure you are using 64 bit EMS binary
- Set max_msg_memory= <XX>GB in your tibemsd.conf, where XX indicated the Memory in GB you wanna assign.
- Set msg_swapping=enabled.
max_msg_memory = size [KB|MB|GB]
Maximum memory the server can use for messages.
This parameter lets you limit the memory that the server uses for messages, so server memory usage cannot grow beyond the system’s memory capacity.
When msg_swapping is enabled, and messages overflow this limit, the server begins to swap messages from process memory to disk. Swapping allows the server to free process memory for incoming messages, and to process message volume in excess of this limit.
When the server swaps a message to disk, a small record of the swapped message remains in memory. If all messages are swapped out to disk, and their remains still exceed this memory limit, then the server has no room for new incoming messages. The server stops accepting new messages, and send calls in message producers result in an error. (This situation probably indicates either a very low value for this parameter, or a very high message volume.)
Specify units as KB, MB or GB. The minimum value is 8 MB. The default value of 0 (zero) indicates no limit.
max_msg_memory = 512MB
msg_swapping = enable | disable
This parameter enables and disables the message swapping feature (described above for max_msg_memory).
The default value is enabled, unless you explicitly set it to disabled.
reserve_memory = size
When reserve_memory is non-zero, the daemon allocates a block of memory for use in emergency situations to prevent the EMS server from being unstable in low memory situations. When the daemon process exhausts memory resources, it disables clients and routes from producing new messages, and frees this block of memory to allow consumers to continue operation (which tends to free memory).
The EMS server attempts to reallocate its reserve memory once the number of pending messages in the server has dropped to 10% of the number of pending messages that were in the server when it experienced the allocation error. If the server successfully reallocates memory, it begins accepting new messages.
The reserve_memory parameter only triggers when the EMS server has run out of memory and therefore is a reactive mechanism. The appropriate administrative action when an EMS server has triggered release of reserve memory is to drain the majority of the messages by consuming them and then to stop and restart the EMS server. This allows the operating system to reclaim all the virtual memory resources that have been consumed by the EMS server. A trace option, MEMORY, is also available to help show what the server is doing during the period when it is not accepting messages.
Specify size in units of MB. When non-zero, the minimum block is 16MB. When absent, the default is zero.