SERVER1 / Search

What are you looking for?

Search services, pages and articles. Results appear after 2 characters.

View all articles
VPS & Cloud

Why did a batch of 500 AI product descriptions stop halfway?

Follow a batch from an API rate limit to one saved draft. Learn how a worker resumes without duplicate descriptions and when a VPS is justified.

SERVER1.GE2 min read
Why did a batch of 500 AI product descriptions stop halfway?

An editor selected 500 products and received 60 drafts. Before restarting the batch, find the last committed result. The browser may have stopped waiting while the worker is still running.

A training incident, minute by minute

09:00. The web request creates a job and returns its number. Product IDs enter a persistent queue. The editor may close the page. A background worker reads one item and submits its catalog data for drafting.

09:12. Sixty drafts are saved. The API returns 429 for the next item. Here it means a rate limit. Record product ID, attempt and status. Respect the provider’s retry delay when supplied, otherwise use increasing delays with jitter. A hard quota or budget exhaustion needs separate handling. API rate limits.

09:14. After a restart, the first 60 items are complete. Product 61 has an uncertain result: the provider may have generated a response just before the connection failed. Automatically inserting a fresh draft for each response could give the editor two versions.

Of four work cards, product 061 returns for processing. Its stable key produces only one draft.
Of four work cards, product 061 returns for processing. Its stable key produces only one draft.

09:15. Identify work by product, source version, language and task, for example SKU-061 / v3 / ka / description. Give the database a uniqueness rule for that combination. The worker claims an item, saves the draft and completed state in one transaction, then acknowledges the job. On redelivery it checks for the committed result. If the source changed, flag it for review. Celery task and acknowledgement semantics.

The editor sees one draft for product 61 and checks material, dimensions and included parts against the catalog before publication. A lost reply can still incur two external calls. Reconcile usage with the provider.

Time and operating limits

500 ÷ 5 = 100 minutes at five usable descriptions per minute. A two-minute pause and ten extra attempts at that pace add four minutes, for an illustrative 104. Human review takes additional time.

A small periodic batch can fit existing hosting. For a late WordPress launch, check WP-Cron. A dedicated VPS becomes worthwhile when you need a persistent worker, supervised restart and queue monitoring. Assign an owner for the backlog, failed jobs, disk and logs. Limit attempts per item and total batch spending. Before running 500, stop a five-item test after saving the third draft. It should finish with exactly five distinct drafts.

SERVER1.GE

Not sure which infrastructure fits this workload?

Tell us what you run, your expected traffic and current setup. We will help you narrow down the right option.

Relevant services
Choose by workload, not by habit.

Use the article context and your actual resource needs to compare the recommended options.