Somchai Chili Radio · Health
Live health recovered
Info
Live endpoint recovered for Somchai Chili Radio
Status: Resolved
Occurrences: 1
Last seen: 2026-08-19 23:27:52 UTC
View raw incident details
Project: Somchai Chili Radio
Domain: 100.67.241.67
Endpoint: Live
URL: http://100.67.241.67:18080/health
Status: 200
Response time: 25ms
Recovered at: 2026-08-19T23:27:52.819568+00:00
Response: {"ok":true,"service":"somchai-api","db":"ok","qdrant":"ok","mode":"isolated-vps-station-manager"}
Somchai Chili Radio · Health
Live health check failed
Info
Live endpoint is failing for Somchai Chili Radio
Status: Resolved
Occurrences: 1
Last seen: 2026-08-19 23:27:52 UTC
View raw incident details
Project: Somchai Chili Radio
Domain: 100.67.241.67
Endpoint: Live
URL: http://100.67.241.67:18080/health
Status: no response
Response time: 19665ms
Consecutive failures: 2
Checked at: 2026-08-19T23:22:07.450153+00:00
Somchai Chili Radio · Log
1 log error(s) detected
Error
- Root cause: AI analysis is temporarily unavailable, so Watchtower is sending a basic fallback summary. - Distinct errors: 1 - First signal: 2026-08-19T23:19:41.961187219Z raise e.OperationalError(str(last_exc)) - Recommended action: review the raw log excerpt below and try the analysis again later if needed. AI error: Cannot connect to host generativelanguage.googleapis.com:443 ssl:<ssl.SSLContext object at 0x71adeb6b2b50> [Temporary failure in name res…
Status: Recorded
Occurrences: 1
Last seen: 2026-08-19 23:24:26 UTC
View full analysis
- Root cause: AI analysis is temporarily unavailable, so Watchtower is sending a basic fallback summary.
- Distinct errors: 1
- First signal: 2026-08-19T23:19:41.961187219Z raise e.OperationalError(str(last_exc))
- Recommended action: review the raw log excerpt below and try the analysis again later if needed.
AI error: Cannot connect to host generativelanguage.googleapis.com:443 ssl:<ssl.SSLContext object at 0x71adeb6b2b50> [Temporary failure in name resolution]
Raw errors:
--- Error 1 ---
2026-08-19T23:19:41.961187219Z raise e.OperationalError(str(last_exc))
2026-08-19T23:19:41.961190124Z psycopg.OperationalError: [Errno -3] Temporary failure in name resolution
View raw incident details
Fingerprint: d2ebc270195b
Occurrences: 1
2026-08-19T23:19:41.961187219Z raise e.OperationalError(str(last_exc))
2026-08-19T23:19:41.961190124Z psycopg.OperationalError: [Errno -3] Temporary failure in name resolution
Somchai Chili Radio · Health
Live health recovered
Info
Live endpoint recovered for Somchai Chili Radio
Status: Resolved
Occurrences: 1
Last seen: 2026-08-02 13:37:41 UTC
View raw incident details
Project: Somchai Chili Radio
Domain: 100.67.241.67
Endpoint: Live
URL: http://100.67.241.67:18080/health
Status: 200
Response time: 22ms
Recovered at: 2026-08-02T13:37:41.080858+00:00
Response: {"ok":true,"service":"somchai-api","db":"ok","qdrant":"ok","mode":"isolated-vps-station-manager"}
Somchai Chili Radio · Health
Live health check failed
Info
Live endpoint is failing for Somchai Chili Radio
Status: Resolved
Occurrences: 1
Last seen: 2026-08-02 13:37:41 UTC
View raw incident details
Project: Somchai Chili Radio
Domain: 100.67.241.67
Endpoint: Live
URL: http://100.67.241.67:18080/health
Status: no response
Response time: 11000ms
Consecutive failures: 2
Checked at: 2026-08-02T13:28:13.079918+00:00
Somchai Chili Radio · Health
Live health recovered
Info
Live endpoint recovered for Somchai Chili Radio
Status: Resolved
Occurrences: 1
Last seen: 2026-08-02 08:36:54 UTC
View raw incident details
Project: Somchai Chili Radio
Domain: 100.67.241.67
Endpoint: Live
URL: http://100.67.241.67:18080/health
Status: 200
Response time: 901ms
Recovered at: 2026-08-02T08:36:54.080916+00:00
Response: {"ok":true,"service":"somchai-api","db":"ok","qdrant":"ok","mode":"isolated-vps-station-manager"}
Somchai Chili Radio · Health
Live health check failed
Info
Live endpoint is failing for Somchai Chili Radio
Status: Resolved
Occurrences: 1
Last seen: 2026-08-02 08:36:54 UTC
View raw incident details
Project: Somchai Chili Radio
Domain: 100.67.241.67
Endpoint: Live
URL: http://100.67.241.67:18080/health
Status: no response
Response time: 11000ms
Consecutive failures: 2
Checked at: 2026-08-02T05:47:40.080924+00:00
Somchai Chili Radio · Log
1 log error(s) detected
Error
### **Error 1: Database Query Syntax Error** * **Root Cause**: The application is sending a malformed SQL query to the database. The syntax `now()-interval 3 hours` is incorrect. The database expects the interval value to be a string, like `interval '3 hours'`. * **What Went Wrong**: The application tried to ask the database for a list of tasks from the last three hours. It wrote this request with a grammatical error that the database didn't understand, s…
Status: Recorded
Occurrences: 1
Last seen: 2026-07-29 01:08:57 UTC
View full analysis
### **Error 1: Database Query Syntax Error**
* **Root Cause**: The application is sending a malformed SQL query to the database. The syntax `now()-interval 3 hours` is incorrect. The database expects the interval value to be a string, like `interval '3 hours'`.
* **What Went Wrong**: The application tried to ask the database for a list of tasks from the last three hours. It wrote this request with a grammatical error that the database didn't understand, so the database rejected the command.
* **Potential Fixes**:
* Locate the code responsible for querying the `runs` table.
* Modify the query to wrap the interval value in quotes: `now() - interval '3 hours'`.
* Check how the application's database library or ORM (Object-Relational Mapper) handles time intervals to ensure it generates correct SQL.
* **Severity**: **HIGH**. This error is likely breaking a feature that displays recent task history to users or administrators.
View raw incident details
Fingerprint: 1ffa417d79b3
Occurrences: 1
2026-07-29T01:08:12.771835311Z 2026-07-29 01:08:12.771 UTC [3583] ERROR: syntax error at or near "3" at character 126
2026-07-29T01:08:12.771864324Z 2026-07-29 01:08:12.771 UTC [3583] STATEMENT: select id,task_type,status,input,result,error,created_at,started_at,finished_at from runs where created_at >= now()-interval 3 hours and task_type like $1 order by created_at
Somchai Chili Radio · Log
1 log error(s) detected
Error
### Error 1: Database Syntax Error * **Root Cause:** The application is sending a malformed SQL query to its database. The database parser encountered a backslash character (`\`) where it was not expected, causing a syntax error. * **What Went Wrong (Simple Terms):** The application tried to give an instruction to the database, but the instruction was written with a grammatical mistake. The database didn't understand the command and refused to execute it.…
Status: Recorded
Occurrences: 1
Last seen: 2026-07-29 01:04:46 UTC
View full analysis
### Error 1: Database Syntax Error
* **Root Cause:** The application is sending a malformed SQL query to its database. The database parser encountered a backslash character (`\`) where it was not expected, causing a syntax error.
* **What Went Wrong (Simple Terms):** The application tried to give an instruction to the database, but the instruction was written with a grammatical mistake. The database didn't understand the command and refused to execute it. This is often caused by user input containing special characters that aren't handled correctly.
* **How to Fix:**
* Find the code that constructs the database query that runs around the time of the error (`2026-07-29 01:04:13 UTC`).
* The best practice is to use parameterized queries (also called prepared statements). This separates the query logic from the data, which prevents this type of error and protects against SQL injection attacks.
* If you cannot use parameterized queries, ensure that all data being inserted into the query string is properly escaped.
* **Severity:** **HIGH**. This indicates a feature is broken for certain inputs and strongly suggests a potential security vulnerability (SQL injection).
View raw incident details
Fingerprint: 50db1b075706
Occurrences: 1
2026-07-29T01:04:13.736744326Z 2026-07-29 01:04:13.736 UTC [2995] ERROR: syntax error at or near "\" at character 69
Somchai Chili Radio · Log
1 log error(s) detected
Fatal
### **Error: Telegram Polling Conflict** * **Root Cause**: A `telegram_polling_conflict` error indicates that more than one process is trying to connect to the Telegram API using the same bot token simultaneously. Telegram only allows one active polling connection per token. * **What Went Wrong**: Your application connects to Telegram to get updates (like a person checking for new mail). This error means another copy of your application (or a different ap…
Status: Recorded
Occurrences: 1
Last seen: 2026-07-28 11:30:33 UTC
View full analysis
### **Error: Telegram Polling Conflict**
* **Root Cause**: A `telegram_polling_conflict` error indicates that more than one process is trying to connect to the Telegram API using the same bot token simultaneously. Telegram only allows one active polling connection per token.
* **What Went Wrong**: Your application connects to Telegram to get updates (like a person checking for new mail). This error means another copy of your application (or a different app entirely) is already connected using the same secret key (bot token). Because the line is busy, this application instance failed to connect after several attempts and shut down its Telegram-related functions.
* **How to Fix**:
1. **Find the duplicate**: Search for and stop any other running instances of the application that use the same Telegram bot token. This could be an old container that didn't shut down properly, a copy running in a staging/development environment, or an unintentional duplicate in production.
2. **Restart**: Once you have confirmed no other processes are using the token, restart the affected container. The log explicitly suggests running `hermes gateway restart`.
* **Severity**: **HIGH**. The log states this is a "Fatal telegram adapter error," meaning the application's ability to send or receive messages via Telegram is completely broken.
View raw incident details
Fingerprint: 3a0a9d3b53a3
Occurrences: 1
2026-07-28T11:29:07.866275128Z ERROR gateway.run: Fatal telegram adapter error (telegram_polling_conflict): Telegram polling could not recover after 5 retries (200s total wait). The previous gateway session is still held open on Telegram's servers, or another process is using the same bot token. To recover: ensure no other Hermes or OpenClaw instance is running with this token, then restart the gateway with 'hermes gateway restart'.
Somchai Chili Radio · Log
1 log error(s) detected
Error
### Error 1: Telegram Connection Conflict * **Root Cause:** More than one application instance is attempting to connect to the Telegram API using the same bot token. Telegram's `getUpdates` method only permits one active polling client at a time. * **What Went Wrong:** The application's connection to its Telegram chatbot was forcibly closed. Another program, likely a duplicate copy of this application, tried to log in as the same bot, causing Telegram to…
Status: Recorded
Occurrences: 1
Last seen: 2026-07-28 11:29:16 UTC
View full analysis
### Error 1: Telegram Connection Conflict
* **Root Cause:** More than one application instance is attempting to connect to the Telegram API using the same bot token. Telegram's `getUpdates` method only permits one active polling client at a time.
* **What Went Wrong:** The application's connection to its Telegram chatbot was forcibly closed. Another program, likely a duplicate copy of this application, tried to log in as the same bot, causing Telegram to reject the connection from this instance. As a result, the application cannot receive messages from or send messages to users on Telegram.
* **How to Fix:**
* Search your infrastructure (e.g., other servers, old deployments, staging environments, developer machines) for another process running with the same Telegram bot token and shut it down.
* Once the conflicting process is stopped, restart this application container to re-establish a clean connection to Telegram.
* **Severity:** HIGH
View raw incident details
Fingerprint: d3524cf80fde
Occurrences: 1
2026-07-28T11:29:03.692093434Z ERROR hermes_plugins.telegram_platform.adapter: [Telegram] Telegram polling could not recover after 5 retries (200s total wait). The previous gateway session is still held open on Telegram's servers, or another process is using the same bot token. To recover: ensure no other Hermes or OpenClaw instance is running with this token, then restart the gateway with 'hermes gateway restart'. Original error: Conflict: terminated by other getUpdates request; make sure that only one bot instance is running
Somchai Chili Radio · Log
1 log error(s) detected
Error
### Error 1: Telegram Message Delivery Failure * **Root Cause:** The application generated a response to a Telegram user, but after processing it for Telegram's specific formatting rules, the final message content was empty. The system that tries to recover from this also failed. * **Simple Explanation:** The app tried to reply to a user on Telegram, but the message it created became blank after being prepared for sending. As a result, nothing was sent to…
Status: Recorded
Occurrences: 1
Last seen: 2026-07-28 08:51:07 UTC
View full analysis
### Error 1: Telegram Message Delivery Failure
* **Root Cause:** The application generated a response to a Telegram user, but after processing it for Telegram's specific formatting rules, the final message content was empty. The system that tries to recover from this also failed.
* **Simple Explanation:** The app tried to reply to a user on Telegram, but the message it created became blank after being prepared for sending. As a result, nothing was sent to the user.
* **To Investigate:**
* Review the code or templates that generate responses for the Telegram platform.
* Check what kind of content is being generated that might be entirely stripped out (e.g., unsupported formatting, special characters, or invalid HTML/Markdown).
* Examine the user interaction that triggered this error to understand what the expected response should have been.
* **Severity:** **HIGH**. The application is failing to communicate with users on a key platform, which is a broken core feature.
View raw incident details
Fingerprint: 6d07a9267ab8
Occurrences: 1
2026-07-28T08:50:35.276048336Z ERROR gateway.platforms.base: [Telegram] response_delivery_dropped: non-empty response (51 chars) produced no delivered message or attachment for 1038876913 (empty after extract, recovery yielded nothing).
Somchai Chili Radio · Log
3 log error(s) detected
Error
### **Error: Scheduled Job Failed on Unpublished Content** * **Root Cause:** The logs show a single distinct error event. A scheduled job, `'Chili Radio daily reel approval preview'`, failed because it tried to process a content reel ("Howard Jones reel") that is in a draft or review state, not a "published" state. The application code correctly stopped the process by raising a `RuntimeError`. * **What Went Wrong:** An automated daily task tried to use a…
Status: Recorded
Occurrences: 1
Last seen: 2026-06-29 02:07:00 UTC
View full analysis
### **Error: Scheduled Job Failed on Unpublished Content**
* **Root Cause:** The logs show a single distinct error event. A scheduled job, `'Chili Radio daily reel approval preview'`, failed because it tried to process a content reel ("Howard Jones reel") that is in a draft or review state, not a "published" state. The application code correctly stopped the process by raising a `RuntimeError`.
* **What Went Wrong:** An automated daily task tried to use a piece of content that was still waiting for approval from a user named "Michael". The system is designed to only work with finished, published content for this task, so it stopped and reported the error.
* **Potential Fixes:**
* **Investigate the Job's Logic:** Check if the job should only be selecting "published" reels. If so, update its query to filter out items in other states.
* **Check the Content Workflow:** Determine why the "Howard Jones reel" has not been published. Is it stuck waiting for review? Does the approval process need attention?
* **Update Application Code:** If the job is actually supposed to handle draft content, the code must be changed to manage this state without failing.
* **Severity:** **HIGH**. A core automated business process is failing, which could impact content pipelines or daily operations.
---
### **Note: Informational Message (Not an Error)**
The log line `Note: the earlier reported 'PIL' error...` is not an error. It is a diagnostic message confirming that a software component for image processing (`PIL`) is installed and working correctly in this environment. No action is required for this message.
View raw incident details
Fingerprint: 75374a1af07b
Occurrences: 1
2026-06-29T02:05:45.247119654Z ERROR cron.scheduler: Job 'Chili Radio daily reel approval preview' failed: RuntimeError: Howard Jones reel preview is ready for Michael review only — not published.
Fingerprint: 2efd059ada1c
Occurrences: 2
2026-06-29T02:05:45.247559675Z Note: the earlier reported `PIL` error appears environment-specific; in this runtime `PIL` is available and the script completed successfully.
Fingerprint: 9059c3c79f63
Occurrences: 1
2026-06-29T02:05:45.247867538Z raise RuntimeError(_err_text)
2026-06-29T02:05:45.247946176Z RuntimeError: Howard Jones reel preview is ready for Michael review only — not published.
Somchai Chili Radio · Log
1 log error(s) detected
Error
### Error 1 * **Root Cause:** The application attempted to send a file via its Telegram integration. However, the code provided the server's root directory (`/`) as the file path instead of a path to an actual file. * **Simple Explanation:** The app tried to send a file to someone on Telegram, but instead of picking a specific file, it mistakenly tried to send the entire server's main folder. A folder cannot be sent as a single file, so the operation fail…
Status: Recorded
Occurrences: 1
Last seen: 2026-06-28 07:31:41 UTC
View full analysis
### Error 1
* **Root Cause:** The application attempted to send a file via its Telegram integration. However, the code provided the server's root directory (`/`) as the file path instead of a path to an actual file.
* **Simple Explanation:** The app tried to send a file to someone on Telegram, but instead of picking a specific file, it mistakenly tried to send the entire server's main folder. A folder cannot be sent as a single file, so the operation failed.
* **Potential Fixes:**
* Investigate the code that generates file paths for the Telegram `sendDocument` function.
* Check for a missing or incorrect configuration value that might be defaulting to `/`.
* Add validation to ensure the path is a file and not a directory before attempting the send operation.
* **Severity:** **HIGH**. A key integration feature (sending documents via Telegram) is completely broken.
View raw incident details
Fingerprint: a149a773d0a1
Occurrences: 1
2026-06-28T07:30:28.023442249Z [Telegram] Failed to send document: [Errno 21] Is a directory: '/'
Somchai Chili Radio · Log
5 log error(s) detected
Error
### **Scheduled Job Failure: 'daily reel approval preview'** * **Root Cause:** The application is missing a required Python library for image processing. The log `ModuleNotFoundError: No module named 'PIL'` indicates that the `Pillow` (or `PIL`) library is not installed in the Docker container's environment. * **Explanation:** An automated daily job, designed to create a visual preview for a "reel," failed. The process requires tools to handle images, but…
Status: Recorded
Occurrences: 1
Last seen: 2026-06-28 02:07:27 UTC
View full analysis
### **Scheduled Job Failure: 'daily reel approval preview'**
* **Root Cause:** The application is missing a required Python library for image processing. The log `ModuleNotFoundError: No module named 'PIL'` indicates that the `Pillow` (or `PIL`) library is not installed in the Docker container's environment.
* **Explanation:** An automated daily job, designed to create a visual preview for a "reel," failed. The process requires tools to handle images, but a key image-processing tool was not found. The system attempted several backup options (like finding photos of different artists), but these also failed, causing the entire job to be aborted.
* **Potential Fixes:**
1. Add `Pillow` to the application's dependency file (e.g., `requirements.txt`).
2. Rebuild and redeploy the Docker image so the missing library is included.
3. Investigate the fallback logic (e.g., why it failed to find a suitable photo for "Bonnie Tyler") to improve the job's resilience.
* **Severity:** **HIGH**. A core automated content generation feature is completely broken.
View raw incident details
Fingerprint: faf6d4edcb8c
Occurrences: 1
2026-06-28T02:06:56.436998826Z ERROR cron.scheduler: Job 'Chili Radio daily reel approval preview' failed: RuntimeError: Reel job did **not** produce a sendable approval preview.
Fingerprint: 89f2a4235fa4
Occurrences: 2
2026-06-28T02:06:56.437036883Z - Initial reported failure was `ModuleNotFoundError: No module named 'PIL'`.
Fingerprint: 1b08eecdca8c
Occurrences: 2
2026-06-28T02:06:56.437043413Z - The generator’s automatic fallback first picked **Bonnie Tyler**, but Commons sourcing failed: no suitable licensed real photo found.
Fingerprint: b2f156e2efe5
Occurrences: 2
2026-06-28T02:06:56.437062973Z - But visual QA failed: the rendered Paul Young image was too dark/cropped/obscured for a recognizable artist preview. I did **not** send it as ready.
Fingerprint: 9059c3c79f63
Occurrences: 1
2026-06-28T02:06:56.437084386Z raise RuntimeError(_err_text)
2026-06-28T02:06:56.437086789Z RuntimeError: Reel job did **not** produce a sendable approval preview.
Somchai Chili Radio · Log
3 log error(s) detected
Error
### **Error: 'Daily reel approval preview' job failure** * **Root Cause:** A scheduled background job (`Chili Radio daily reel approval preview`) is failing because it is incorrectly programmed to treat a successful outcome as a critical error. The application successfully generated a preview but then raised a `RuntimeError`, which stopped the job. The log message mentioning a "Pillow/PIL dependency" appears to be a misleading or outdated message and not…
Status: Recorded
Occurrences: 1
Last seen: 2026-06-27 02:07:07 UTC
View full analysis
### **Error: 'Daily reel approval preview' job failure**
* **Root Cause:** A scheduled background job (`Chili Radio daily reel approval preview`) is failing because it is incorrectly programmed to treat a successful outcome as a critical error. The application successfully generated a preview but then raised a `RuntimeError`, which stopped the job. The log message mentioning a "Pillow/PIL dependency" appears to be a misleading or outdated message and not the cause of this specific failure.
* **What Went Wrong (in simple terms):** A daily automated task that creates content previews is working correctly, but the code is written in a way that it panics and reports a major error when it finishes its job. It's like a chef successfully baking a cake and then smashing the fire alarm because the cake is ready.
* **To Investigate / Fix:**
1. Review the code for the `Chili Radio daily reel approval preview` job.
2. Locate where the `RuntimeError` is being raised and change the logic to handle this "ready for approval" status as a success instead of an error.
3. Find and remove the incorrect log message about "Pillow/PIL" to avoid confusion during future debugging.
* **Severity: HIGH**
* This is a failing background job that appears to be part of a core content creation or approval workflow. Its failure likely prevents new content from being published.
View raw incident details
Fingerprint: 52c21d972a32
Occurrences: 1
2026-06-27T02:06:24.948860275Z ERROR cron.scheduler: Job 'Chili Radio daily reel approval preview' failed: RuntimeError: Stevie Nicks Artist Flashback preview is ready for approval — not published.
Fingerprint: 85ab685ce56b
Occurrences: 2
2026-06-27T02:06:24.948938282Z - Recovered the cron failure by installing the missing Pillow/PIL dependency, then rerendered.
Fingerprint: 9059c3c79f63
Occurrences: 1
2026-06-27T02:06:24.949033906Z raise RuntimeError(_err_text)
2026-06-27T02:06:24.949041207Z RuntimeError: Stevie Nicks Artist Flashback preview is ready for approval — not published.
Somchai Chili Radio · Log
8 log error(s) detected
Error
### Analysis of Telegram Timeout Error * **Root Cause:** The application, a Telegram bot, failed to get a timely response from Telegram's servers when checking for new messages. The network request timed out. * **What Went Wrong (Simple Explanation):** The application tried to "call" the Telegram service to ask "Are there any new messages for me?". It waited for an answer, but the call took too long, so the application hung up. This usually happens becaus…
Status: Recorded
Occurrences: 1
Last seen: 2026-06-27 01:12:47 UTC
View full analysis
### Analysis of Telegram Timeout Error
* **Root Cause:** The application, a Telegram bot, failed to get a timely response from Telegram's servers when checking for new messages. The network request timed out.
* **What Went Wrong (Simple Explanation):** The application tried to "call" the Telegram service to ask "Are there any new messages for me?". It waited for an answer, but the call took too long, so the application hung up. This usually happens because of a slow network connection or a problem with the Telegram service itself.
* **Suggested Investigation:**
* **Check Network:** Verify the container's internet connection is stable and that it can reach `api.telegram.org`. Look for firewall, DNS, or proxy issues.
* **Increase Timeout:** As the error message suggests, increase the network request timeout value in the application's code. This will make it wait longer for a response.
* **Check Telegram Status:** Investigate if the Telegram API service was experiencing an outage or performance degradation at the time of the error.
* **Container Resources:** Ensure the container and its host machine are not overloaded (high CPU/memory), as this can slow down network operations.
* **Severity:** **HIGH**
* This error prevents the bot from receiving new messages, which is a core function.
View raw incident details
Fingerprint: 2fcaa99d0a34
Occurrences: 1
2026-06-27T01:11:50.417792996Z ERROR telegram.ext.Updater: Error while calling `get_updates` one more time to mark all fetched updates. Suppressing error to ensure graceful shutdown. When polling for updates is restarted, updates may be fetched again. Please adjust timeouts via `ApplicationBuilder` or the parameter `get_updates_request` of `Bot`.
Fingerprint: 36e8bc933d57
Occurrences: 1
2026-06-27T01:11:50.417862129Z File "/usr/local/lib/hermes-agent/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 101, in map_httpcore_exceptions
Fingerprint: 44e967631a14
Occurrences: 1
2026-06-27T01:11:50.418056132Z with map_exceptions(exc_map):
Fingerprint: 8f26258cbc50
Occurrences: 1
2026-06-27T01:11:50.418070845Z File "/usr/local/lib/hermes-agent/venv/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
Fingerprint: a5c23a180d0c
Occurrences: 2
2026-06-27T01:11:50.418090584Z The above exception was the direct cause of the following exception:
Fingerprint: 98e23d61968d
Occurrences: 1
2026-06-27T01:11:50.418228312Z with map_httpcore_exceptions():
Fingerprint: 94a94f8d02b9
Occurrences: 1
2026-06-27T01:11:50.418243024Z File "/usr/local/lib/hermes-agent/venv/lib/python3.11/site-packages/httpx/_transports/default.py", line 118, in map_httpcore_exceptions
Fingerprint: ade1136149c2
Occurrences: 1
2026-06-27T01:11:50.418479401Z telegram.error.TimedOut: Timed out
Somchai Chili Radio · Log
3 log error(s) detected
Error
### **Analysis of 'Daily Reel Approval Preview' Job Failure** * **Root Cause:** The scheduled job to create a social media reel could not find source images for the intended artists. The system then used a fallback artist ("Billy Idol") to generate the content. As a safety measure, the code is intentionally designed to raise this error to halt the process and prevent the automatic publishing of unvetted fallback content. * **What Went Wrong (in simple ter…
Status: Recorded
Occurrences: 1
Last seen: 2026-06-26 02:06:32 UTC
View full analysis
### **Analysis of 'Daily Reel Approval Preview' Job Failure**
* **Root Cause:** The scheduled job to create a social media reel could not find source images for the intended artists. The system then used a fallback artist ("Billy Idol") to generate the content. As a safety measure, the code is intentionally designed to raise this error to halt the process and prevent the automatic publishing of unvetted fallback content.
* **What Went Wrong (in simple terms):** The daily automated task that creates a video post couldn't find the right pictures. It used a backup plan to make a video about the artist "Billy Idol" instead. The system then stopped itself on purpose to ask a human to check this backup video before it gets posted online.
* **Potential Fixes / Investigation:**
* Investigate why the image sourcing failed for the original artists. Check the connection to the image source (the logs mention "Commons") and whether suitable images exist for the intended artists.
* Manually review the generated "Billy Idol" reel to decide if it should be published.
* Consider changing this "error" into a non-failing notification (e.g., an email or Slack alert) that content is awaiting manual approval, to avoid triggering unnecessary alarms.
* **Severity:** **MEDIUM**
* The application is not down, but a key automated content pipeline is interrupted and now requires manual intervention. The system correctly prevented a potential issue, but the intended daily content was not produced as expected.
View raw incident details
Fingerprint: 8ab8065982c5
Occurrences: 1
2026-06-26T02:05:31.217281295Z ERROR cron.scheduler: Job 'Chili Radio daily reel approval preview' failed: RuntimeError: Billy Idol reel generated for approval.
Fingerprint: d4a26b777806
Occurrences: 2
2026-06-26T02:05:31.217399142Z Also fixed the initial cron failure cause: Pillow/PIL is available now. The script initially failed on Commons sourcing for a couple of random fallback artists, then succeeded with unused artist Billy Idol. No Facebook/Instagram publishing was done.
Fingerprint: 9059c3c79f63
Occurrences: 1
2026-06-26T02:05:31.217408446Z raise RuntimeError(_err_text)
2026-06-26T02:05:31.217411200Z RuntimeError: Billy Idol reel generated for approval.
Somchai Chili Radio · Log
2 log error(s) detected
Error
### Error: Misconfigured Error in 'daily reel' Job * **Root Cause:** The logs show a single issue from a scheduled task named `'Chili Radio daily reel approval preview'`. The task is being forced to fail with a `RuntimeError`, but the message itself (`Draft reel created for approval: Phil Collins.`) describes a successful operation. This indicates a programming mistake where a success case is incorrectly being reported as a critical error. * **Explanation…
Status: Recorded
Occurrences: 1
Last seen: 2026-06-25 02:09:13 UTC
View full analysis
### Error: Misconfigured Error in 'daily reel' Job
* **Root Cause:** The logs show a single issue from a scheduled task named `'Chili Radio daily reel approval preview'`. The task is being forced to fail with a `RuntimeError`, but the message itself (`Draft reel created for approval: Phil Collins.`) describes a successful operation. This indicates a programming mistake where a success case is incorrectly being reported as a critical error.
* **Explanation:** A background job that creates daily "reels" is running. It appears to be successfully creating a draft for an artist named "Phil Collins". However, the program is coded to treat this successful outcome as a failure, which causes an error alert to be generated unnecessarily.
* **Suggested Fix:**
* A developer should review the code for the `'Chili Radio daily reel approval preview'` job.
* The line that raises the `RuntimeError` should be changed to log a standard informational message instead of a critical error.
* **Severity:** **MEDIUM**
* While the application is not down, this scheduled job is likely failing every time it runs. This could prevent important follow-up actions (like sending notifications or finalizing the reel) from occurring, breaking a core feature.
View raw incident details
Fingerprint: ff6baa8ccf9a
Occurrences: 1
2026-06-25T02:08:40.228774305Z ERROR cron.scheduler: Job 'Chili Radio daily reel approval preview' failed: RuntimeError: Draft reel created for approval: Phil Collins.
Fingerprint: 9059c3c79f63
Occurrences: 1
2026-06-25T02:08:40.228844380Z raise RuntimeError(_err_text)
2026-06-25T02:08:40.228846443Z RuntimeError: Draft reel created for approval: Phil Collins.
Somchai Chili Radio · Log
1 log error(s) detected
Error
**Error 1: Insufficient Funds for External Service** * **Root Cause:** The application is making a request to a third-party, paid service to perform a "Compression summary". The account for this service has run out of funds or credits, causing the service to reject the request with a `402 Payment Required` error. * **What Went Wrong:** The app tried to use an external tool that it pays for, but the payment failed because the account is empty. The tool ref…
Status: Recorded
Occurrences: 1
Last seen: 2026-06-24 02:17:17 UTC
View full analysis
**Error 1: Insufficient Funds for External Service**
* **Root Cause:** The application is making a request to a third-party, paid service to perform a "Compression summary". The account for this service has run out of funds or credits, causing the service to reject the request with a `402 Payment Required` error.
* **What Went Wrong:** The app tried to use an external tool that it pays for, but the payment failed because the account is empty. The tool refused to do the work. The application handled this by inserting a placeholder ("fallback context marker") instead of the real summary.
* **How to Fix:**
* Identify the external service being called (e.g., an AI/LLM provider, a data processing API).
* Add funds or credits to the account associated with that service.
* Set up billing alerts with the service provider to prevent this from happening in the future.
* **Severity:** HIGH. A core feature is non-functional, leading to a degraded user experience, even though the application did not crash.
View raw incident details
Fingerprint: 298602277575
Occurrences: 1
2026-06-24T02:16:22.035634713Z ⚠ Compression summary failed: Error code: 402 - {'type': 'error', 'error': {'type': 'insufficient_balance_error', 'message': 'insufficient balance (1008)'}, 'request_id': '068a70f5fbff64b140e1243386230810'}. Inserted a fallback context marker.