...

What is an Iguana Crash?

• An unexpected and abrupt shutdown of the Iguana process or service.

...

Things to ask the client

• Are there crash dump files in the Iguana working directory? If so, send them to us, following the documentation here

• Iguana services are running but unable to process messages

• Iguana can process messages but unable to access dashboard

• Iguana services and dashboard are working but unable to commit changes

• Iguana service repeatedly crashes when trying to start the service

Preliminary Investigation

To better understand and investigate Iguana crash, you can prepare the following steps prior reaching out to iNTERFACEWARE support:

• Crash Dump: When Iguana unexpected shutdown, it will generate a crash report called crash dump. If this report successfully generated in your Iguana working directory, you will be asked to send these crash dump to an secure FTP server for iNTERFACEWARE development team to analysis: http://help.interfaceware.com/v6/how-to-report-problems#ftp

  • Before asking the client to send us the crash dump files, make sure they are running Iguana 6. The development team will not look at Iguana 5 crash dump files.

  • Some factors to consider if no crashdumps were created:

    • The server ran out of memory

    • Iguana does not have the correct permissions

    • Too many opened files, cannot open a new file for crashdump

    • mac and Linux might have it disabled more information here

• Send us the ServiceErrorLog.txt file

  Look for errors caused by the

  ServiceErrorLog.txt: Upon Iguana failure, ServiceErrorLog will be created to document the common Iguana errors. You can review this file and explore the common Iguana errors (ex. log files being locked by another program

  .

• In order to aid the development team in their investigation, also ask the following:

  • Operating system, process architecture, and Iguana version

  • What they were doing when the crash happened, and how many users were using Iguana at that time?

  • Were there any recent changes to the Iguana environment?

  • Is there any backup or antivirus software on their server?

  • Is there any scheduled server maintenance, and if so, does this involve a server restart?

  • Is this a production instance?

• If the instance is HA

  • Search the company in Google Drive and understand their HA architecture diagram

  • Did WK1 or WK2 go down?

  • Did the failover execute successfully?

Common crashes

• Logs locked

• Ran out of memory

• Node.lua

• An old Iguana instance may still be running after an improper shutdown (potentially due to server maintenance), and can have a lock on the log files still

• Git lock file

• Corrupted Iguana)

• Preparation Questions: Here is the list of investigation questions that could help you either to find root cause or alternative solutions

  • Operating system and Iguana version: help to identify if crash related some OS or particular Iguana version

  • Last a few activities prior to the Crash: help to quick identify the possible crash root causes

  • Recent changes to the Iguana or environment: help to include/exclude external factors

  • Recent backup or antivirus software to the Iguana Server: common Iguana crash scenario

  • Recent server maintenance (ex. server restart): common Iguana crash scenario

  • Prior to Crash Memory and CPU Usage: common Iguana crash scenario

  • Iguana Types (ex. Production vs QA): for production Iguana Crash, bringing Iguana back online has higher priority than finding the root cause

  • (Enterprise Customer) Iguana High Availability: additional information that will help to determine if HA tools are causing the Iguana crash

Common crashes

If above preliminary investigation could not help you to resolve your Iguana crash, here are some common Crash Scenarios and Best Practices;

...

Adapted from Paul Le’s article in Basecamp.