System Troubleshooting & Incident Response Runbook

This runbook provides actionable, step-by-step diagnostic and remediation protocols for Eniceberny Bakery and Culinary Hub. It is structured for counter cashiers, kitchen dispatchers, administrative managers, customer support agents, and DevOps engineers.

---

🚨 Quick Incident Triage Matrix

graph TD
    Issue[Operational Incident Detected] --> Cat{Identify Category}
    Cat -->|Counter / Register| POS[1. POS & Cashier Runbook]
    Cat -->|Payment / MoMo| PAY[2. Payment & Gateway Runbook]
    Cat -->|Kitchen / Courier| DISP[3. Kitchen & Delivery Runbook]
    Cat -->|Customer / Login| CUST[4. Customer & Account Runbook]
    Cat -->|Admin / Settings| ADM[5. Admin Back-Office Runbook]
    Cat -->|Server / DevOps| OPS[6. DevOps & Engineering Runbook]

---

1. Point of Sale (POS) Counter & Cashier Runbook

1.1 Thermal Receipt Printer Unresponsive / Paper Jam

• Symptoms: Cashier clicks "Complete Sale & Print Receipt", but the thermal printer remains silent, or paper feeds without printing text.
• Immediate Cashier Action:
  1. Check printer power light. If off, verify the AC power adapter and cable connection to the wall outlet.
  2. Open the printer cover latch. Inspect the 80mm thermal paper roll:
     • Ensure the paper roll is inserted with the paper feeding from the bottom (thermal coating faces down).
     • Remove any crumpled or jammed paper.
     • Close the latch firmly until it clicks.
  3. In the browser pop-up print dialog, verify the destination printer is set to the thermal printer (e.g. POS-80 or Thermal Receipt Printer), not "Save as PDF".
  4. If the print dialog was accidentally closed, open the order record in /admin/orders and click "Print Receipt" to re-trigger the receipt print.

1.2 Cash Tender Arithmetic or Wrong Change Input

• Symptoms: Cashier typed an incorrect cash tender amount (e.g. entered GH₵ 500 instead of GH₵ 50) and completed the transaction.
• Immediate Action:
  1. The receipt will show the erroneous tender and change calculation, but the order's actual total collected and product inventory remain accurate.
  2. Re-calculate the correct change manually: $$\text{Actual Change} = \text{Actual Cash Handed} - \text{Order Total}$$
  3. Hand the patron the correct physical change immediately with an apology.
  4. Make an immediate written note in the Shift Cash Log detailing the Order Number and the arithmetic difference so the Shift Supervisor accounts for it during drawer balancing at shift close.

1.3 Cash Drawer Does Not Pop Open Automatically

• Symptoms: Receipt prints successfully, but the physical cash drawer fails to kick open.
• Resolution:
  1. Check the RJ-11 / RJ-12 telephone-style kick cable connecting the bottom of the receipt printer to the back of the cash drawer. Ensure it is firmly seated.
  2. Use the physical manual emergency key under the drawer chassis to release the lock.
  3. Ensure the key slot on the front of the cash drawer is turned to the vertical (electronic release) position rather than the horizontal (deadlocked) position.

1.4 POS Screen Freezes or Network Blip

• Symptoms: Clicking product cards does not add them to the ticket, or the search bar is non-responsive.
• Resolution:
  1. Press Ctrl + Shift + R (or Cmd + Shift + R on Mac) to perform a hard browser reload and clear cached assets.
  2. The POS system persists operational assets locally; verify your local Wi-Fi or Ethernet connection to the store router.
  3. If the terminal was mid-checkout, check /admin/orders to confirm whether the ticket went through before re-entering it to prevent duplicate sales.

---

2. Mobile Money (MoMo) & Payment Gateway Runbook

2.1 Customer Does Not Receive MTN MoMo USSD Prompt

• Symptoms: Customer selects Mobile Money at checkout, but their mobile phone does not display the prompt to enter their PIN.
• Root Causes:
  • Telecom USSD channel congestion.
  • Insufficient customer MoMo wallet balance.
  • Incorrect phone number entered (e.g. missing digit or international format mismatch).
• Remediation Steps:
  1. Check Phone Format: Ensure the number begins with 0 (e.g. 024XXXXXXX) or +233.
  2. Manual USSD Push: Advise the customer to manually check their pending transaction approvals:
     • MTN MoMo: Dial *170# -> Select Option 6 (My Wallet) -> Option 3 (My Approvals) -> Enter PIN -> Approve the Eniceberny transaction.
     • Telecel Cash: Dial *110# -> Select Option 5 (My Account) -> Option 4 (Approvals).
  3. If no approval exists within 2 minutes, instruct the patron to refresh /checkout and initiate a fresh prompt, or select Cash on Delivery / In-Store Collection.

2.2 Customer Charged Twice for a Single Order

• Symptoms: Customer states their telecom wallet was debited twice due to double-clicking the checkout button.
• Verification & Refund Protocol:
  1. Navigate to /admin/payments and search by the customer's phone number or name.
  2. Inspect the payment_transactions ledger for duplicate references.
  3. Log into the official Paystack Dashboard (https://dashboard.paystack.com):
     • Search transactions by reference or phone.
     • If two successful charges exist for the same order, locate the duplicate transaction and click "Refund".
     • Enter the reason: "Customer accidental duplicate checkout - Order EB-XXXX".
  4. The refunded funds will be returned to the customer's MoMo wallet within 24 to 48 hours according to telecom banking rules.

2.3 Paystack Webhook Not Updating Order Status

• Symptoms: Customer received debit SMS, but the order in /admin/orders remains in payment_status = pending.
• Admin Manual Verification:
  1. Open /admin/orders/{id} for the affected order.
  2. Copy the Paystack payment reference from the transaction table.
  3. Click the "Verify with Gateway" button on the order page.
  4. The backend queries https://api.paystack.co/transaction/verify/{reference} directly. If Paystack returns status = 'success', the order updates immediately to payment_status = paid, changes order status to confirmed, and records the audit log.

---

3. Kitchen Fulfillment & Delivery Dispatch Runbook

3.1 Delivery Courier No-Show or Delayed Pickup

• Symptoms: Order has been in ready_for_pickup for more than 15 minutes without an assigned dispatch rider arriving.
• Action Plan:
  1. Open /admin/orders/{id} and check the assigned delivery service and contact telephone.
  2. Call the dispatch company hotline immediately.
  3. If the primary courier service cannot dispatch within 10 minutes, escalate to our backup dispatch partner (e.g. Bolt Food Business or Swift Courier).
  4. Keep the food stored in thermal insulated hot-boxes or refrigeration (for cakes/salads) to preserve culinary temperature.
  5. Call the customer to proactively inform them:

     "Hello [Customer Name], this is Eniceberny Bakery. Your fresh order is hot and packaged in our kitchen. We are expediting a priority rider right now and will update you the moment they depart."

3.2 Incorrect Delivery Address or Missing GhanaPostGPS

• Symptoms: Dispatch rider calls the kitchen stating they cannot locate the customer's building or landmark.
• Action Plan:
  1. Open /admin/orders/{id} to review the full delivery_address, gps_address, and special_instructions.
  2. If the GPS address is provided (e.g. AK-039-2311), copy it into the GhanaPostGPS mobile app or Google Maps to pinpoint the exact latitude/longitude.
  3. Send the GPS coordinates and pin directly to the courier via WhatsApp.
  4. If coordinates are absent or invalid, call the customer's primary phone number and secondary contact immediately to request a live WhatsApp location pin.

3.3 Food Transit Damage or Accidental Spill

• Symptoms: Courier drops package or cake frosting is damaged in transit.
• Customer Recovery Protocol:
  1. Apologize profusely and prioritize customer delight above all else.
  2. Instruct the kitchen to immediately begin an Expedited Priority Re-Make.
  3. Dispatch the replacement order via a dedicated express vehicle at zero cost to the customer.
  4. Include a complimentary baker's treat (e.g. a box of fresh gourmet meat pies or a bottle of artisanal Sobolo) with an apology card.
  5. Document the incident in /admin/orders/{id} notes and log the damaged items in /admin/inventory as waste_spoilage.

---

4. Customer Account & Storefront Runbook

4.1 Customer Does Not Receive Email Verification OTP

• Symptoms: Customer registers an account, but the 6-digit email verification code does not arrive in their inbox.
• Resolution:
  1. Instruct the customer to inspect their Spam, Junk, or Promotions folder for an email from orders@enicebakerygh.com.
  2. Customers can click "Resend Verification Code" on /verify-email after a 60-second cooldown period.
  3. Admin Intervention:
     • If the customer is on the phone or at the counter, a Store Administrator can navigate to /admin/customers, locate the customer's account, and click "Manually Verify Customer".
     • This sets email_verified_at = now() and activates the account instantly.

4.2 Cart Drawer Items Disappear or Do Not Persist

• Symptoms: Customer adds pastries to the shopping cart, but refreshing the page shows an empty basket.
• Root Cause: Third-party cookies or session storage disabled in browser, or cross-domain cookie blocking in Safari (ITP).
• Resolution:
  1. Check browser cookie settings: Ensure cookies are enabled for enicebakerygh.com.
  2. Clear browser cache and cookies for the domain.
  3. Note that guest carts expire after 7 days of inactivity. Logging in to a customer account permanently binds active cart items to the database user_id.

---

5. Administration & Back-Office Operations Runbook

5.1 Image Upload Fails with "413 Payload Too Large" or Format Error

• Symptoms: Admin tries to upload a product photo, hero banner, or brand logo, and the upload errors out.
• Resolution:
  1. File Size Limit: Uploaded media must be under 5 MB. If the file is 10–20 MB straight from a professional DSLR, compress it using an image optimizer (e.g. TinyPNG or Squoosh) before uploading.
  2. Allowed Formats: Supported file extensions are .jpg, .jpeg, .png, .webp, and .svg. Unsupported formats like .heic, .raw, or .bmp must be converted to .jpg or .webp.
  3. PHP Server Directive: DevOps can verify upload_max_filesize = 20M and post_max_size = 25M in docker/php/php.ini.

5.2 Settings Changes Do Not Appear on Public Storefront

• Symptoms: Admin updates store hotline, delivery fees, or site announcement in /admin/settings, but visitors still see old information.
• Root Cause: Laravel application configuration and Blade view caching.
• Remediation:
  1. Settings are cached in Redis / file storage for high-speed performance (cache_for: 3600 seconds).
  2. In the Admin settings screen, click the "Purge Application Cache" button in Card 8.
  3. Or run the Artisan CLI command:

     php artisan cache:clear && php artisan config:clear && php artisan view:clear
     

5.3 CSV Export Freezes or Times Out on Large Date Ranges

• Symptoms: Clicking "Export CSV" on /admin/reports with a 1-year date range results in a 504 Gateway Timeout.
• Resolution:
  1. The report controller utilizes streamed chunked responses (cursor() and fputcsv()) to minimize RAM consumption.
  2. For exceptionally large historical datasets (10,000+ orders), narrow the filter to a 1-month or 3-month window before exporting.

---

6. DevOps, Infrastructure & Server Runbook

6.1 PostgreSQL Connection Limit Exhausted

• Symptoms: Web application throws PDOException: SQLSTATE[08006] [7] FATAL: remaining connection slots are reserved for non-replication superuser connections.
• Immediate Resolution:
  1. Check connection pool configuration in .env / config/database.php.
  2. On the PostgreSQL host, inspect active connections:

     SELECT pid, usename, client_addr, state, query_start, query 
     FROM pg_stat_activity 
     WHERE state != 'idle';
     

  3. Terminate idle or orphan connections:

     SELECT pg_terminate_backend(pid) 
     FROM pg_stat_activity 
     WHERE state = 'idle' 
       AND state_change < current_timestamp - INTERVAL '5 minutes';
     

  4. Ensure connection pooling (e.g. PgBouncer on Render / AWS RDS) is active with max_connections = 100 and application connections pooled.

6.2 Docker Container Healthcheck Failure (/up)

• Symptoms: Container enters crash loop or Render healthcheck fails on HTTP endpoint /up.
• Diagnostics:
  1. Inspect container logs:

     docker logs -f <container_id> --tail 100
     

  2. Verify directory permissions on storage and bootstrap cache:

     chown -R www-data:www-data /var/www/html/storage /var/www/html/bootstrap/cache
     chmod -R 775 /var/www/html/storage /var/www/html/bootstrap/cache
     

  3. Test health check response manually:

     curl -i http://localhost:8080/up
     

     Should return HTTP/1.1 200 OK.

6.3 GitBook Git Sync Fails or Conflicts

• Symptoms: Updates pushed to main branch are not reflected on https://docs.enicebakerygh.com.
• Diagnostic Steps:
  1. Verify root configuration file gitbook-docs.yaml exists and matches schema https://api.gitbook.com/gitbook-docs.yaml.
  2. Ensure all markdown files referenced in docs/SUMMARY.md actually exist inside ./docs.
  3. In the GitBook space settings, check Integrations -> Git Sync:
     • Verify GitHub repository permissions and webhook delivery status.
     • Click "Resync Now" to trigger a manual pull from GitHub main.

---

For architectural specifications or database schemas, refer to the System Architecture Guide and Database Schema Guide.