diff --git a/AGENTS.md b/AGENTS.md index afce89bf6..9d2040654 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -47,7 +47,7 @@ After `pnpm run build`, run `mcpc` (no args) to check sessions: if `@stdio` (def - **Integration tests**: `pnpm run test:integration` (needs build + `APIFY_TOKEN`, humans only). - **Conformance tests**: `pnpm run test:conformance` (needs `APIFY_TOKEN`) builds and runs both protocol eras locally. `_integration_tests.yaml` calls `_conformance_tests.yaml` for the same `2026-07-28` and `2025-11-25` coverage in CI. Both eras always run; the command exits with the first non-zero code. Scenarios excluded from each era's gate are listed with reasons in `scripts/conformance_expected_failures_2026_07_28.yaml` and `scripts/conformance_expected_failures_2025_11_25.yaml`. - **Package manager**: this repo uses **pnpm 11+**. `devEngines.packageManager` is pinned with `onFail: "error"`, so npm / yarn refuse to run inside the checkout — use `pnpm install` only. -- `tests/integration/suite.ts` wires the `stdio`, `2025-11-25` (streamable HTTP) and `2026-07-28` (stateless HTTP) transport dimensions into one shared suite. Add new integration cases to the matching capability array in `tests/test_kit/cases/*.cases.ts` (registration, tools, actors, apps, tasks, storage, schedules, payments) — each is a plain `Case[]` (`{ name, isDeploymentTest, run, ... }`), registered via `registerCases(name, cases, ctx)`. Mark a case `isDeploymentTest: true` only if it's worth running against `apify-mcp-server-internal`'s live staging and production deploys — the flag doesn't just share the case, it makes internal execute it for real on every release (see [DEVELOPMENT.md](./DEVELOPMENT.md#test-organization-across-repos)). Shared assertion helpers live in `tests/test_kit/helpers.ts`. Cases that share expensive setup (e.g. one seeded Actor run, see `storage.cases.ts`'s `normalModeRunFixture`) do so via `ctx.getFixture(fixture)`, memoized once per transport dimension — not a vitest `beforeAll`, so any subset of cases can share it while staying flat `Case` objects. +- `tests/integration/suite.ts` wires the `stdio`, `2025-11-25` (streamable HTTP) and `2026-07-28` (stateless HTTP) transport dimensions into one shared suite. Add new integration cases to the matching capability array in `tests/test_kit/cases/*.cases.ts` (registration, tools, actors, apps, tasks, storage, schedules, versions, payments) — each is a plain `Case[]` (`{ name, isDeploymentTest, run, ... }`), registered via `registerCases(name, cases, ctx)`. Mark a case `isDeploymentTest: true` only if it's worth running against `apify-mcp-server-internal`'s live staging and production deploys — the flag doesn't just share the case, it makes internal execute it for real on every release (see [DEVELOPMENT.md](./DEVELOPMENT.md#test-organization-across-repos)). Shared assertion helpers live in `tests/test_kit/helpers.ts`. Cases that share expensive setup (e.g. one seeded Actor run, see `storage.cases.ts`'s `normalModeRunFixture`) do so via `ctx.getFixture(fixture)`, memoized once per transport dimension — not a vitest `beforeAll`, so any subset of cases can share it while staying flat `Case` objects. - Follow existing test patterns (names, structure) — check neighboring files. - **Test naming**: `describe('fnName()')`, plain-verb `it()` names (no `should` prefix). Group with nested `describe()` per method when a factory/class exposes several. diff --git a/README.md b/README.md index 7015417b5..78acb193b 100644 --- a/README.md +++ b/README.md @@ -249,7 +249,7 @@ Here are some special MCP operations and how the Apify MCP Server supports them: - **Actor tasks**: Create, inspect, and update your saved Actor tasks, and publish or unpublish their public landing pages. - **Schedules**: Create, inspect, update, and delete schedules that run your Actors and tasks automatically. - **Builds**: Build an Actor version, list builds, check the status of a build, and retrieve its build log. -- **Actor source**: Read an Actor version's files, with a hash for each file and a revision for the whole set. +- **Actor versions**: Read an Actor version's files with a hash for each file and a revision for the whole set, create an Actor from files, and change a version's files with checks that catch changes made since your last read. - **Apify API**: Search the Apify API reference, inspect an operation, and send requests to the API to read or change data. ### Overview of available tools @@ -298,7 +298,9 @@ Legend for the **Enabled by default** column: | `get-actor-build-log` | builds | Retrieve the logs for a specific Actor build. | | | `get-actor-build-list` | builds | List the account's builds, or one Actor's, in every status, newest first. | | | `build-actor` | builds | Build an Actor version and wait a bounded time for the build to finish. | | -| `get-actor-version` | source | Get an Actor version's file listing with hashes and the content of the files you ask for. | | +| `get-actor-version` | versions | Get an Actor version's file listing with hashes and the content of the files you ask for. | | +| `create-actor` | versions | Create a private Actor in your account from files, optionally starting a build. | | +| `update-actor-version` | versions | Write, edit, or delete files of an Actor version in one all-or-nothing write. | | | `apify-api-search` | api | Search the Apify API reference for operations by keywords. Returns each one's method, path, and docs page. | | | `apify-api-details` | api | Get the parameters and request body schema of the Apify API operations on a path. | | | `apify-api-read` | api | Send a GET request to an Apify API path, such as `actor-runs` or `/v2/actor-runs`, like `apify api` in the Apify CLI. Read-only: it refuses the `method` query parameter and a GET to the synchronous run endpoints, which starts a paid run. | | @@ -424,6 +426,7 @@ The v2 configuration preserves backward compatibility with v1 usage. Notes: - `call-actor` is now included by default via the `actors` category (additive change). To exclude it, specify an explicit `tools` list without `actors`. - `tools=add-actor`, `tools=experimental`, and `tools=preview` are retired: they are ignored and load no tools. Use `tools=call-actor` (or the default `actors` category) instead. - `tools=get-actor-log` is retired: the tool was renamed to `get-actor-run-log`. The old selector is ignored and loads no tools. Use `tools=get-actor-run-log` (or the `runs` category) instead. +- `tools=source` is retired: the category was renamed to `versions`. The old selector is ignored and loads no tools. Use `tools=versions` instead. Existing URLs and commands using `?actors=...` or `--actors` continue to work unchanged. diff --git a/src/const.ts b/src/const.ts index c291f8a53..dbc473f53 100644 --- a/src/const.ts +++ b/src/const.ts @@ -54,6 +54,7 @@ export const HELPER_TOOLS = { ACTOR_BUILD_LOG: 'get-actor-build-log', ACTOR_BUILD_LIST_GET: 'get-actor-build-list', ACTOR_CALL: 'call-actor', + ACTOR_CREATE: 'create-actor', ACTOR_CALL_WIDGET: 'call-actor-widget', ACTOR_DELETE: 'delete-actor', ACTOR_GET_DETAILS: 'fetch-actor-details', @@ -70,6 +71,7 @@ export const HELPER_TOOLS = { ACTOR_TASK_PUBLISH: 'publish-actor-task', ACTOR_TASK_UNPUBLISH: 'unpublish-actor-task', ACTOR_VERSION_GET: 'get-actor-version', + ACTOR_VERSION_UPDATE: 'update-actor-version', DATASET_GET: 'get-dataset', DATASET_LIST_GET: 'get-dataset-list', DATASET_GET_ITEMS: 'get-dataset-items', @@ -96,16 +98,17 @@ export type HelperToolName = (typeof HELPER_TOOLS)[keyof typeof HELPER_TOOLS]; /** * Retired tool selectors: `add-actor` and `experimental` (add-actor was deleted in the stateless - * migration), the deprecated `preview` pseudo-category, and `get-actor-log` (renamed to - * `get-actor-run-log`). They name neither a registry category nor a real tool anymore, so they - * resolve to nothing — never loaded, never treated as an Actor ID, never requiring a token by - * themselves. + * migration), the deprecated `preview` pseudo-category, `get-actor-log` (renamed to + * `get-actor-run-log`), and the `source` category (renamed to `versions`). They name neither a + * registry category nor a real tool anymore, so they resolve to nothing — never loaded, never + * treated as an Actor ID, never requiring a token by themselves. */ export const RETIRED_SELECTOR_NAMES: ReadonlySet = new Set([ 'add-actor', 'experimental', 'preview', 'get-actor-log', + 'source', ]); /** @@ -283,6 +286,7 @@ export const APIFY_ERROR_TYPE_MEMORY_LIMIT_EXCEEDED = 'memory-limit-exceeded'; export const APIFY_ERROR_TYPE_CANNOT_START_ACTOR_RUNS = 'cannot-start-actor-runs'; export const APIFY_ERROR_TYPE_CANNOT_PUBLISH_ACTOR_TASK = 'cannot-publish-actor-task'; export const APIFY_ERROR_TYPE_INVALID_INPUT = 'invalid-input'; +export const APIFY_ERROR_TYPE_ACTOR_NAME_NOT_UNIQUE = 'actor-name-not-unique'; export const APIFY_ERROR_TYPE_PAGE_NOT_FOUND = 'page-not-found'; // HTTP status codes diff --git a/src/tools/AGENTS.md b/src/tools/AGENTS.md index 8f7adeb09..85d0153c8 100644 --- a/src/tools/AGENTS.md +++ b/src/tools/AGENTS.md @@ -32,16 +32,20 @@ direct actor tools, `search-actors`, `fetch-actor-details`) is mode-agnostic. - `tasks/` — Actor task create/get/update plus publish/unpublish of the task's public landing page (`task_helpers.ts` holds the shared task response shape and the publication call). - `schedules/` — schedule create/get/update/delete for Actors and tasks; `schedule_helpers.ts` - converts the flat action shape to the API shape and back, and reuses the id helpers from + converts the flat action shape to the API shape and back, and reuses `getTaskByIdOrName` from `tasks/task_helpers.ts`. - `builds/` — `get-actor-build` (build status), `get-actor-build-log` (build log tail), `get-actor-build-list` (the account's builds, or one Actor's with `actorId`, in every status, newest first, pointing at the newest failed one) and `build-actor` (start a build of one version and wait for it); `build_helpers.ts` holds the allowlisted build result shape, the build start and wait calls (the wait reports progress), the shared `waitSecs` field, the shared build response and the by-status next-step text. - - `source/`: `get-actor-version` (an Actor version's file manifest with hashes, revision, and - requested content). `source_files.ts` holds the file shape, the manifest builder, the hash and - revision rules, and the text-or-base64 detection. Versions not stored as files (Git repository, + - `versions/`: `get-actor-version` (an Actor version's file manifest with hashes, revision, and + requested content), `create-actor` (a new Actor with one version in a single POST), and + `update-actor-version` (write, edit, and delete operations applied all-or-nothing to a fresh read, + stored with one version PUT that carries only the source keys). `source_files.ts` holds the file + shape, the manifest builder, the hash and revision rules, and the text-or-base64 detection; + `source_helpers.ts` holds the Actor and version lookup, the file input shape, and the response + after a write, which starts the build for autoBuild. Versions not stored as files (Git repository, gist, or zip) are refused. - `api/` — the generic Apify API tools: search the operations of the published OpenAPI spec, get the operations on a path, and send a GET, or a POST, PUT, PATCH, or DELETE, to a path. The write tool is in @@ -76,6 +80,9 @@ direct actor tools, `search-actors`, `fetch-actor-details`) is mode-agnostic. - `docs/` — search and fetch Apify docs. - `dev/` — the `report-problem` tool for reporting a problem with a tool or Actor. - `widgets/` — the `*-widget` tool variants (apps mode only). +- `../utils/apify_resource_id.ts` holds the helpers that resolve an Actor, task, or schedule given by ID or + name, where a name alone means one of the caller's own. The task, schedule, and versions tools use them, + and so do `build-actor` and the `actorId` filter of `get-actor-build-list` and `get-actor-run-list`. ## Rules when editing here diff --git a/src/tools/builds/build_actor.ts b/src/tools/builds/build_actor.ts index af20496ef..ff93377b0 100644 --- a/src/tools/builds/build_actor.ts +++ b/src/tools/builds/build_actor.ts @@ -4,6 +4,7 @@ import { HELPER_TOOLS } from '../../const.js'; import type { InternalToolArgs, ToolDescriptionContext, ToolEntry, ToolInputSchema } from '../../types.js'; import { ALL_TOOLS_PRESENT, TOOL_TYPE } from '../../types.js'; import { compileSchema, fixZodSchemaRequired } from '../../utils/ajv.js'; +import { getResourceByIdOrName } from '../../utils/apify_resource_id.js'; import { getConsoleLinkContext } from '../../utils/console_link.js'; import { respondAborted, respondUserError } from '../../utils/mcp.js'; import { ABORT } from '../actors/actor_run_response.js'; @@ -18,7 +19,12 @@ import { } from './build_helpers.js'; const buildActorArgs = z.object({ - actor: z.string().min(1).describe('Actor ID or username/name'), + actor: z + .string() + .min(1) + .describe( + 'The Actor to build: its ID, its name (resolved against your own Actors), or its full name as username/name or username~name.', + ), // Format is not enforced by a regex: AJV here drops `pattern`, and the lookup against the Actor's // versions below rejects anything that is not an existing MAJOR.MINOR version with a soft fail. versionNumber: z @@ -74,9 +80,12 @@ export const buildActor: ToolEntry = Object.freeze({ call: async (toolArgs: InternalToolArgs) => { const { args, apifyClient: client, apifyToken, loadedToolNames, signal, progressTracker } = toolArgs; const parsed = buildActorArgs.parse(args); - const actor = await client.actor(parsed.actor).get(); - if (!actor) { - return respondUserError(`Actor '${parsed.actor}' not found.`); + const actor = await getResourceByIdOrName(parsed.actor, async (safeId) => client.actor(safeId).get()); + // Extra path segments, such as username/name/runs/last, reach a sub-resource that is not an Actor. + if (!actor || typeof actor.name !== 'string' || typeof actor.username !== 'string') { + return respondUserError( + `Actor '${parsed.actor}' not found. The actor argument takes an ID, a name (resolved against your own Actors), or a full name as username/name.`, + ); } const versionNumbers = listVersionNumbers(actor); if (versionNumbers.length === 0) { diff --git a/src/tools/builds/get_actor_build_list.ts b/src/tools/builds/get_actor_build_list.ts index bd770ec47..afc16e471 100644 --- a/src/tools/builds/get_actor_build_list.ts +++ b/src/tools/builds/get_actor_build_list.ts @@ -5,6 +5,7 @@ import { HELPER_TOOLS } from '../../const.js'; import type { InternalToolArgs, ToolDescriptionContext, ToolEntry, ToolInputSchema } from '../../types.js'; import { ALL_TOOLS_PRESENT, TOOL_TYPE } from '../../types.js'; import { compileSchema, fixZodSchemaRequired } from '../../utils/ajv.js'; +import { getResourceByIdOrName } from '../../utils/apify_resource_id.js'; import { respondOk, respondUserError } from '../../utils/mcp.js'; import { catchNotFound } from '../storage/storage_helpers.js'; import { getActorBuildListToolOutputSchema } from '../structured_output_schemas.js'; @@ -16,7 +17,7 @@ const getActorBuildListArgs = z.object({ .min(1) .optional() .describe( - 'Only list builds of this Actor: its ID, or its full name as username/name or username~name. A name without the username is not enough.', + 'Only list builds of this Actor: its ID, its name (resolved against your own Actors), or its full name as username/name or username~name.', ), offset: z.number().int().min(0).describe('Number of builds to skip at the start. Default: 0.').default(0), limit: z @@ -116,11 +117,17 @@ export const getActorBuildList: ToolEntry = Object.freeze({ const { args, apifyClient: client, loadedToolNames } = toolArgs; const parsed = getActorBuildListArgs.parse(args); const listOptions = { offset: parsed.offset, limit: parsed.limit, desc: parsed.desc }; + // The list endpoint takes the Actor as `~name` too, so a name is resolved without a separate Actor read. const builds = parsed.actorId - ? await catchNotFound(client.actor(parsed.actorId).builds().list(listOptions)) + ? await getResourceByIdOrName( + parsed.actorId, + async (safeId) => (await catchNotFound(client.actor(safeId).builds().list(listOptions))) ?? undefined, + ) : await client.builds().list(listOptions); if (!builds) { - return respondUserError(`Actor '${parsed.actorId}' not found.`); + return respondUserError( + `Actor '${parsed.actorId}' not found. The actorId argument takes an ID, a name (resolved against your own Actors), or a full name as username/name.`, + ); } // Cast because the client's item type omits fields the API returns (see `BuildListItem`). const items = builds.items as BuildListItem[]; diff --git a/src/tools/registry.ts b/src/tools/registry.ts index ed2e9d8c6..babcfb09a 100644 --- a/src/tools/registry.ts +++ b/src/tools/registry.ts @@ -41,7 +41,6 @@ import { createSchedule } from './schedules/create_schedule.js'; import { deleteSchedule } from './schedules/delete_schedule.js'; import { getSchedule } from './schedules/get_schedule.js'; import { updateSchedule } from './schedules/update_schedule.js'; -import { getActorVersion } from './source/get_actor_version.js'; import { getDataset } from './storage/get_dataset.js'; import { getDatasetItems } from './storage/get_dataset_items.js'; import { getDatasetList } from './storage/get_dataset_list.js'; @@ -55,6 +54,9 @@ import { getActorTask } from './tasks/get_actor_task.js'; import { publishActorTask } from './tasks/publish_actor_task.js'; import { unpublishActorTask } from './tasks/unpublish_actor_task.js'; import { updateActorTask } from './tasks/update_actor_task.js'; +import { createActor } from './versions/create_actor.js'; +import { getActorVersion } from './versions/get_actor_version.js'; +import { updateActorVersion } from './versions/update_actor_version.js'; import { callActorWidget } from './widgets/call_actor_widget.js'; import { fetchActorDetailsWidget } from './widgets/fetch_actor_details_widget.js'; import { getActorRunWidget } from './widgets/get_actor_run_widget.js'; @@ -78,7 +80,7 @@ export const toolCategories = { tasks: [createActorTask, getActorTask, updateActorTask, publishActorTask, unpublishActorTask], schedules: [createSchedule, getSchedule, updateSchedule, deleteSchedule], builds: [getActorBuild, getActorBuildLog, getActorBuildList, buildActor], - source: [getActorVersion], + versions: [getActorVersion, createActor, updateActorVersion], api: [apifyApiSearch, apifyApiDetails, apifyApiRead], dev: [reportProblem], } satisfies Record; diff --git a/src/tools/runs/get_actor_run_list.ts b/src/tools/runs/get_actor_run_list.ts index d06ae6485..54490e3f7 100644 --- a/src/tools/runs/get_actor_run_list.ts +++ b/src/tools/runs/get_actor_run_list.ts @@ -4,6 +4,7 @@ import { HELPER_TOOLS } from '../../const.js'; import type { InternalToolArgs, ToolEntry, ToolInputSchema } from '../../types.js'; import { TOOL_TYPE } from '../../types.js'; import { compileSchema } from '../../utils/ajv.js'; +import { getResourceByIdOrName } from '../../utils/apify_resource_id.js'; import { respondRaw, respondUserError } from '../../utils/mcp.js'; import { catchNotFound } from '../storage/storage_helpers.js'; import { actorRunListOutputSchema } from '../structured_output_schemas.js'; @@ -14,7 +15,7 @@ const getUserRunsListArgs = z.object({ .min(1) .optional() .describe( - 'Only list runs of this Actor: its ID, or its full name as username/name or username~name. A name without the username is not enough.', + 'Only list runs of this Actor: its ID, its name (resolved against your own Actors), or its full name as username/name or username~name.', ), offset: z .number() @@ -79,11 +80,18 @@ USAGE EXAMPLES: desc: parsed.desc, status: parsed.status, }; + // The list endpoint takes the Actor as `~name` too, so a name is resolved without a separate Actor read. const runs = parsed.actorId - ? await catchNotFound(apifyClient.actor(parsed.actorId).runs().list(listOptions)) + ? await getResourceByIdOrName( + parsed.actorId, + async (safeId) => + (await catchNotFound(apifyClient.actor(safeId).runs().list(listOptions))) ?? undefined, + ) : await apifyClient.runs().list(listOptions); if (!runs) { - return respondUserError(`Actor '${parsed.actorId}' not found.`); + return respondUserError( + `Actor '${parsed.actorId}' not found. The actorId argument takes an ID, a name (resolved against your own Actors), or a full name as username/name.`, + ); } return respondRaw({ content: [{ type: 'text', text: JSON.stringify(runs) }], diff --git a/src/tools/schedules/schedule_helpers.ts b/src/tools/schedules/schedule_helpers.ts index 4a2121a88..2ab07c7db 100644 --- a/src/tools/schedules/schedule_helpers.ts +++ b/src/tools/schedules/schedule_helpers.ts @@ -9,9 +9,10 @@ import { import { z } from 'zod'; import type { ApifyClient } from '../../apify_client.js'; +import { getResourceByIdOrName } from '../../utils/apify_resource_id.js'; import { toIsoString } from '../actors/actor_run_response.js'; import { anyJsonObjectSchema } from '../structured_output_schemas.js'; -import { getResourceByIdOrName, getTaskByIdOrName } from '../tasks/task_helpers.js'; +import { getTaskByIdOrName } from '../tasks/task_helpers.js'; /** A schedule action as the API accepts it on create and update (the server-assigned `id` is optional). */ type ApiScheduleAction = NonNullable[number]; diff --git a/src/tools/schedules/update_schedule.ts b/src/tools/schedules/update_schedule.ts index 30dbadbd4..bc38d850b 100644 --- a/src/tools/schedules/update_schedule.ts +++ b/src/tools/schedules/update_schedule.ts @@ -5,9 +5,9 @@ import { HELPER_TOOLS } from '../../const.js'; import type { InternalToolArgs, ToolDescriptionContext, ToolEntry, ToolInputSchema } from '../../types.js'; import { ALL_TOOLS_PRESENT, TOOL_TYPE } from '../../types.js'; import { compileSchema } from '../../utils/ajv.js'; +import { isAmbiguousResourceId, toSafeResourceId } from '../../utils/apify_resource_id.js'; import { respondOk, respondUserError } from '../../utils/mcp.js'; import { scheduleOutputSchema } from '../structured_output_schemas.js'; -import { isAmbiguousResourceId, toSafeResourceId } from '../tasks/task_helpers.js'; import { buildApiActions, buildScheduleResult, diff --git a/src/tools/structured_output_schemas.ts b/src/tools/structured_output_schemas.ts index c6c74b709..cd2658839 100644 --- a/src/tools/structured_output_schemas.ts +++ b/src/tools/structured_output_schemas.ts @@ -634,6 +634,68 @@ export const getActorVersionToolOutputSchema = { required: ['actorId', 'fullName', 'versionNumber', 'revision', 'files', 'contents'], }; +/** The build a source write started with autoBuild: the same allowlisted subset (`toBuildResult`) as get-actor-build. */ +const sourceWriteBuildProperties = { + build: getActorBuildToolOutputSchema.properties.build, + buildError: { + type: 'string', + description: 'Why the build did not start; set when autoBuild was true and the write succeeded without a build', + }, + warnings: { + type: 'array', + items: { type: 'string' }, + description: 'Things to know about the written files, such as empty files the build skips', + }, +}; + +/** Schema for create-actor: the new Actor, its one version, and the files as the platform stored them. */ +export const createActorToolOutputSchema = { + type: 'object' as const, + properties: { + actorId: { type: 'string', description: 'ID of the new Actor' }, + fullName: { type: 'string', description: 'Actor full name, username/name' }, + versionNumber: { type: 'string', description: 'Version number, e.g. 0.0' }, + revision: { + type: 'string', + description: 'Identifies the stored file set; the same value a later read of the version returns', + }, + files: { + type: 'array', + description: 'The files as stored, sorted by path', + items: getActorVersionToolOutputSchema.properties.files.items, + }, + ...sourceWriteBuildProperties, + }, + required: ['actorId', 'fullName', 'versionNumber', 'revision', 'files'], +}; + +/** Schema for update-actor-version: the new revision and what changed. */ +export const updateActorVersionToolOutputSchema = { + type: 'object' as const, + properties: { + revision: { + type: 'string', + description: 'Revision after the update, the value expectedRevision takes in a later update', + }, + changed: { type: 'boolean', description: 'False when the operations changed nothing, so nothing was written' }, + changes: { + type: 'array', + description: 'One entry per file the operations created, updated, or deleted, sorted by path', + items: { + type: 'object', + properties: { + path: { type: 'string', description: 'Path relative to the Actor root' }, + action: { type: 'string', enum: ['created', 'updated', 'deleted'] }, + hash: { type: 'string', description: 'Hash of the file after the update; absent for deleted' }, + }, + required: ['path', 'action'], + }, + }, + ...sourceWriteBuildProperties, + }, + required: ['revision', 'changed', 'changes'], +}; + // Per-storage entry shapes. Factories (not shared constants) because `structuredClone` preserves // object identity: if `default` and `additionalProperties` referenced the same object, cloning // `actorRunOutputSchema` would keep them as the same object, and injecting `itemsSchema` into diff --git a/src/tools/tasks/create_actor_task.ts b/src/tools/tasks/create_actor_task.ts index 8d4791037..f202594b3 100644 --- a/src/tools/tasks/create_actor_task.ts +++ b/src/tools/tasks/create_actor_task.ts @@ -5,9 +5,10 @@ import { HELPER_TOOLS } from '../../const.js'; import type { InternalToolArgs, ToolDescriptionContext, ToolEntry, ToolInputSchema } from '../../types.js'; import { ALL_TOOLS_PRESENT, TOOL_TYPE } from '../../types.js'; import { compileSchema } from '../../utils/ajv.js'; +import { toSafeResourceId } from '../../utils/apify_resource_id.js'; import { respondOk } from '../../utils/mcp.js'; import { actorTaskOutputSchema, anyJsonObjectSchema } from '../structured_output_schemas.js'; -import { publicConfigSchema, taskNameSchema, taskResult, toSafeResourceId } from './task_helpers.js'; +import { publicConfigSchema, taskNameSchema, taskResult } from './task_helpers.js'; const createActorTaskArgs = z.object({ actorId: z diff --git a/src/tools/tasks/task_helpers.ts b/src/tools/tasks/task_helpers.ts index 2058ef1dd..76dc7f597 100644 --- a/src/tools/tasks/task_helpers.ts +++ b/src/tools/tasks/task_helpers.ts @@ -2,6 +2,7 @@ import type { Task, TaskPublicConfig } from 'apify-client'; import { z } from 'zod'; import type { ApifyClient } from '../../apify_client.js'; +import { getResourceByIdOrName, isAmbiguousResourceId, toSafeResourceId } from '../../utils/apify_resource_id.js'; import { toIsoString } from '../actors/actor_run_response.js'; const PUBLIC_CONFIG_FIELDS = [ @@ -12,42 +13,6 @@ const PUBLIC_CONFIG_FIELDS = [ 'datasetView', ] as const satisfies (keyof Omit)[]; -/** Apify resource IDs are exactly 17 alphanumeric characters. */ -const APIFY_ID_REGEX = /^[a-zA-Z0-9]{17}$/; - -/** - * The API reads an unqualified id as an ID, so a bare resource name has to be prefixed with `~` to be - * resolved against the authenticated user's own resources. Ids that already carry a username, in either - * the `username/name` or `username~name` format, and ids that are already IDs, are returned unchanged. - * - * Applies to tasks, the Actor a task is created for, and schedules, which the API resolves the same way. - */ -export function toSafeResourceId(idOrName: string): string { - const trimmed = idOrName.trim(); - if (trimmed.includes('/') || trimmed.includes('~')) return trimmed; - return APIFY_ID_REGEX.test(trimmed) ? trimmed : `~${trimmed}`; -} - -/** A bare 17-alnum value is shape-identical to an ID and to a legal task name. */ -export function isAmbiguousResourceId(idOrName: string): boolean { - const trimmed = idOrName.trim(); - return !trimmed.includes('/') && !trimmed.includes('~') && APIFY_ID_REGEX.test(trimmed); -} - -/** - * Fetches a resource by ID, name, or `username/name` through `fetch`, which receives the API-safe id. - * An ambiguous value cannot be resolved by inspection, so it is read as an ID first and, on a miss, - * retried as the caller's own resource name. - */ -export async function getResourceByIdOrName( - idOrName: string, - fetch: (safeId: string) => Promise, -): Promise { - const resource = await fetch(toSafeResourceId(idOrName)); - if (resource || !isAmbiguousResourceId(idOrName)) return resource; - return fetch(`~${idOrName.trim()}`); -} - /** Fetches a task by ID, name, or `username/name`. */ export async function getTaskByIdOrName(client: ApifyClient, idOrName: string): Promise { return getResourceByIdOrName(idOrName, async (safeId) => client.task(safeId).get()); diff --git a/src/tools/tasks/update_actor_task.ts b/src/tools/tasks/update_actor_task.ts index 02547c92a..19c0ebc92 100644 --- a/src/tools/tasks/update_actor_task.ts +++ b/src/tools/tasks/update_actor_task.ts @@ -5,16 +5,10 @@ import { HELPER_TOOLS } from '../../const.js'; import type { InternalToolArgs, ToolDescriptionContext, ToolEntry, ToolInputSchema } from '../../types.js'; import { ALL_TOOLS_PRESENT, TOOL_TYPE } from '../../types.js'; import { compileSchema } from '../../utils/ajv.js'; +import { isAmbiguousResourceId, toSafeResourceId } from '../../utils/apify_resource_id.js'; import { respondOk, respondUserError } from '../../utils/mcp.js'; import { actorTaskOutputSchema, anyJsonObjectSchema } from '../structured_output_schemas.js'; -import { - getTaskByIdOrName, - isAmbiguousResourceId, - publicConfigSchema, - taskNameSchema, - taskResult, - toSafeResourceId, -} from './task_helpers.js'; +import { getTaskByIdOrName, publicConfigSchema, taskNameSchema, taskResult } from './task_helpers.js'; const updateActorTaskArgs = z.object({ taskId: z diff --git a/src/tools/versions/create_actor.ts b/src/tools/versions/create_actor.ts new file mode 100644 index 000000000..64c4a92f9 --- /dev/null +++ b/src/tools/versions/create_actor.ts @@ -0,0 +1,194 @@ +import type { ActorCollectionCreateOptions, ActorVersionSourceFile } from 'apify-client'; +import { ActorSourceType, ApifyApiError } from 'apify-client'; +import dedent from 'dedent'; +import { z } from 'zod'; + +import { APIFY_ERROR_TYPE_ACTOR_NAME_NOT_UNIQUE, HELPER_TOOLS } from '../../const.js'; +import { UserInputError } from '../../errors.js'; +import type { InternalToolArgs, ToolEntry, ToolInputSchema } from '../../types.js'; +import { TOOL_TYPE } from '../../types.js'; +import { compileSchema, fixZodSchemaRequired } from '../../utils/ajv.js'; +import { respondAborted, respondUserError } from '../../utils/mcp.js'; +import { createActorToolOutputSchema } from '../structured_output_schemas.js'; +import { buildFilesManifest, buildFilesRevision } from './source_files.js'; +import { + buildEmptyFilesWarnings, + buildSourceFileEntry, + extractSourceFiles, + redactSourceToolArgs, + resolveVersion, + respondAfterWrite, + respondToSourceToolError, + SourcePathIndex, + sourceFileArgs, + validateFileContent, + validateFilePath, + validatePathCharacters, +} from './source_helpers.js'; + +const createActorArgs = z.object({ + name: z.string().min(1).describe('Name of the new Actor, without a username, for example my-scraper.'), + title: z.string().min(1).optional().describe('Display title, for example My Scraper.'), + description: z.string().optional().describe('A short description of what the Actor does.'), + files: z.array(sourceFileArgs).min(1).describe("The Actor's files, each with its whole content."), + versionNumber: z + .string() + .optional() + .describe( + 'Number of the version in MAJOR.MINOR form. Default: the version in .actor/actor.json, or 0.0 when it has none.', + ), + buildTag: z + .string() + .min(1) + .optional() + .describe('Tag that builds of the version get. The platform uses latest when it is omitted.'), + autoBuild: z + .boolean() + .default(false) + .describe('Start a build of the version after creating the Actor, and return without waiting. Default: false.'), +}); + +/** + * The entries to send, one per file; throws `UserInputError` for a path no file can be written at, and for a path + * given twice, since the platform puts the Actor name in the first .actor/actor.json and the build uses the last. + */ +function buildSourceFileEntries(files: readonly z.infer[]): ActorVersionSourceFile[] { + const entries: ActorVersionSourceFile[] = []; + const pathIndex = new SourcePathIndex([]); + for (const [index, file] of files.entries()) { + validatePathCharacters(file.path, `files[${index}]`); + const entry = buildSourceFileEntry(file); + const label = `files[${index}] (${file.path})`; + validateFilePath(entry.name, label); + validateFileContent(entry, label); + if (pathIndex.hasFile(entry.name)) { + throw new UserInputError(`${label} repeats the path ${entry.name}; each path can be sent only once.`); + } + pathIndex.validateNewFilePath(entry.name, label); + pathIndex.addFile(entry.name); + entries.push(entry); + } + return entries; +} + +/** + * The `version` of .actor/actor.json when it is a non-empty string in valid JSON, as `apify push` reads it. A leading + * byte order mark, which some editors write, is skipped, since JSON.parse refuses it. + */ +function readActorJsonVersion(entries: readonly ActorVersionSourceFile[]): string | undefined { + const actorJson = entries.find(({ name, format }) => name === '.actor/actor.json' && format === 'TEXT'); + if (!actorJson) return undefined; + try { + const { version } = JSON.parse(actorJson.content.replace(/^\uFEFF/, '')) as { version?: unknown }; + return typeof version === 'string' && version !== '' ? version : undefined; + } catch { + return undefined; + } +} + +/** A refusal of the name as taken on a retry of the POST, which an earlier attempt of the same call may have caused. */ +function isRetriedNameTaken(error: unknown): error is ApifyApiError { + return error instanceof ApifyApiError && error.type === APIFY_ERROR_TYPE_ACTOR_NAME_NOT_UNIQUE && error.attempt > 1; +} + +/** + * https://docs.apify.com/api/v2/actors-post + * /v2/actors + * + * One POST creates the Actor and its version together, private and in the token's account. apify-client retries the + * POST after a network error, a timeout, a 429, or a 5xx, so when the platform saved an attempt it did not answer, the + * retry fails on the name that attempt took, and the call says the Actor may exist. + */ +export const createActor: ToolEntry = Object.freeze({ + type: TOOL_TYPE.INTERNAL, + name: HELPER_TOOLS.ACTOR_CREATE, + title: 'Create Actor', + description: dedent` + Create a new private Actor in your account, with one version holding the files you give. It never changes an existing Actor: the platform refuses a name your account already uses. + - files: every file with its path and content. Binary files take base64 content with encoding base64; files with a binary extension such as .png default to it. + The platform sets the name field of .actor/actor.json to the Actor name, so the result lists each file's hash and the version's revision as stored. + autoBuild starts a build after creating the Actor and returns without waiting. + + USAGE: + - Use to upload new Actor code to the Apify platform for the first time. + + USAGE EXAMPLES: + - user_input: Create an Actor called hacker-news-scraper from these files`, + // `fixZodSchemaRequired` strips `autoBuild` from `required` because it has a default. + inputSchema: fixZodSchemaRequired(z.toJSONSchema(createActorArgs)) as ToolInputSchema, + outputSchema: createActorToolOutputSchema, + ajvValidate: compileSchema(z.toJSONSchema(createActorArgs)), + annotations: { + title: 'Create Actor', + readOnlyHint: false, + destructiveHint: false, + // A second call with the same name is refused rather than repeated. + idempotentHint: false, + // autoBuild starts a build, like build-actor. + openWorldHint: true, + }, + redactArgs: (args: Record) => + redactSourceToolArgs(args, { + keys: ['name', 'title', 'description', 'versionNumber', 'buildTag', 'autoBuild'], + listKey: 'files', + entryKeys: ['path', 'encoding'], + }), + call: async (toolArgs: InternalToolArgs) => { + const { args, apifyClient: client, signal, loadedToolNames } = toolArgs; + const parsed = createActorArgs.parse(args); + // A cancel before the POST creates nothing; per the MCP spec the cancelled request gets no response. + if (signal?.aborted) return respondAborted(); + try { + const sourceFiles = buildSourceFileEntries(parsed.files); + // The platform refuses a version without a number, so this defaults to what `apify push` uses. + // AJV turns a null versionNumber into an empty string, which counts as not given. + const versionNumber = parsed.versionNumber || readActorJsonVersion(sourceFiles) || '0.0'; + const created = await client.actors().create({ + name: parsed.name, + ...(parsed.title !== undefined && { title: parsed.title }), + ...(parsed.description !== undefined && { description: parsed.description }), + versions: [ + { + versionNumber, + ...(parsed.buildTag !== undefined && { buildTag: parsed.buildTag }), + sourceType: ActorSourceType.SourceFiles, + sourceFiles, + }, + ], + } satisfies ActorCollectionCreateOptions); + const fullName = `${created.username}/${created.name}`; + // The platform sets the name in .actor/actor.json on create, so the stored files, not the sent ones, match + // a later read. + const version = resolveVersion(created, undefined, fullName); + const files = buildFilesManifest( + extractSourceFiles(version, `Version ${version.versionNumber} of ${fullName}`), + ); + const emptyPaths = files.filter(({ sizeBytes }) => sizeBytes === 0).map(({ path }) => path); + return await respondAfterWrite({ + toolArgs, + autoBuild: parsed.autoBuild, + target: { actorId: created.id, versionNumber: version.versionNumber }, + structuredContent: { + actorId: created.id, + fullName, + versionNumber: version.versionNumber, + revision: buildFilesRevision(files), + files: files.map(({ path, sizeBytes, hash }) => ({ path, sizeBytes, hash })), + ...buildEmptyFilesWarnings(emptyPaths), + }, + summary: `Created the private Actor ${fullName}.`, + }); + } catch (error) { + if (!isRetriedNameTaken(error)) return respondToSourceToolError(error); + const readTool = loadedToolNames.includes(HELPER_TOOLS.ACTOR_VERSION_GET) + ? `; ${HELPER_TOOLS.ACTOR_VERSION_GET} reads its files` + : ''; + return respondUserError( + `The platform refused the name ${parsed.name} as taken when the request was retried, so an earlier ` + + `attempt of this call may have created the Actor ${parsed.name}. ` + + `If it holds the files you sent, this call created it${readTool}.`, + { httpStatus: error.statusCode }, + ); + } + }, +} as const); diff --git a/src/tools/source/get_actor_version.ts b/src/tools/versions/get_actor_version.ts similarity index 61% rename from src/tools/source/get_actor_version.ts rename to src/tools/versions/get_actor_version.ts index 97b88f843..f4c14c67d 100644 --- a/src/tools/source/get_actor_version.ts +++ b/src/tools/versions/get_actor_version.ts @@ -1,7 +1,5 @@ import { posix } from 'node:path'; -import type { Actor, ActorVersion } from 'apify-client'; -import { ActorSourceType } from 'apify-client'; import dedent from 'dedent'; import { z } from 'zod'; @@ -11,10 +9,10 @@ import type { InternalToolArgs, ToolEntry, ToolInputSchema } from '../../types.j import { TOOL_TYPE } from '../../types.js'; import { compileSchema } from '../../utils/ajv.js'; import { respondOk, respondUserError } from '../../utils/mcp.js'; -import { listVersionNumbers } from '../builds/build_helpers.js'; import { getActorVersionToolOutputSchema } from '../structured_output_schemas.js'; import type { SourceFile } from './source_files.js'; import { buildFilesManifest, buildFilesRevision } from './source_files.js'; +import { extractSourceFiles, fetchActor, resolveVersion, respondToSourceToolError } from './source_helpers.js'; const INLINE_LIMIT_KIB = MAX_INLINE_BYTES / 1024; @@ -23,7 +21,7 @@ const getActorVersionArgs = z.object({ .string() .min(1) .describe( - 'The Actor to read: its ID, or its full name as username/name or username~name. A name without the username is not enough.', + 'The Actor to read: its ID, its name (resolved against your own Actors), or its full name as username/name or username~name.', ), // Format is not enforced by a regex: the lookup against the Actor's versions rejects anything that is not an // existing version with a soft fail. @@ -66,91 +64,6 @@ type ReturnedContent = { totalLines?: number; }; -/** The requested version, or the only one when none is requested; throws `UserInputError` otherwise. */ -function resolveVersion( - actor: Pick, - requestedVersionNumber: string | undefined, - actorSelector: string, -): ActorVersion & { versionNumber: string } { - const versionNumbers = listVersionNumbers(actor); - if (requestedVersionNumber === undefined && versionNumbers.length !== 1) { - // The source type and build tag tell the caller which version holds the code it is after. - const versions = actor.versions - .filter((version) => version.versionNumber !== undefined) - .map(({ versionNumber, sourceType, buildTag }) => { - const tag = buildTag ? `, build tag ${buildTag}` : ''; - return `${versionNumber} (${sourceType}${tag})`; - }); - throw new UserInputError(`Specify versionNumber; this Actor has versions: ${versions.join(', ')}.`); - } - const versionNumber = requestedVersionNumber ?? versionNumbers[0]; - const version = actor.versions.find((candidate) => candidate.versionNumber === versionNumber); - if (!version) { - throw new UserInputError( - `Actor '${actorSelector}' has no version ${versionNumber}; available versions: ${versionNumbers.join(', ')}.`, - ); - } - return { ...version, versionNumber }; -} - -/** - * The URL without what can grant access to it: the query string, the password, and, for http and https, the user. An - * SSH user such as `git@` is not a secret and stays. A URL the parser cannot read, such as - * `git@github.com:user/repo.git`, loses only its query string. - */ -function formatUrlWithoutSecrets(url: string): string { - if (URL.canParse(url)) { - const parsed = new URL(url); - parsed.search = ''; - parsed.password = ''; - if (parsed.protocol === 'http:' || parsed.protocol === 'https:') { - parsed.username = ''; - } - return parsed.href; - } - const queryIndex = url.indexOf('?'); - if (queryIndex === -1) return url; - const hashIndex = url.indexOf('#', queryIndex); - return url.slice(0, queryIndex) + (hashIndex === -1 ? '' : url.slice(hashIndex)); -} - -/** - * Why a version not stored as files is refused, and what to use instead. A version keeps the URL fields of a source - * type it used before, so only the URL of its current type is named. - */ -function formatSourceTypeRefusal(version: ActorVersion, versionLabel: string): string { - if (version.sourceType === ActorSourceType.GitRepo) { - const url = version.gitRepoUrl ? ` ${formatUrlWithoutSecrets(version.gitRepoUrl)}` : ''; - return `${versionLabel} has its files in the Git repository${url}, not stored on Apify, so this tool cannot work on them; use the repository.`; - } - if (version.sourceType === ActorSourceType.GitHubGist) { - const url = version.gitHubGistUrl ? ` ${formatUrlWithoutSecrets(version.gitHubGistUrl)}` : ''; - return `${versionLabel} has its files in the GitHub gist${url}, not stored on Apify, so this tool cannot work on them; use the gist.`; - } - // TODO(#1452): Read zip-stored (TARBALL) versions with adm-zip 0.6.1 or later, within the download and unpacking - // limits the issue lists. - if (version.sourceType === ActorSourceType.Tarball) { - return `${versionLabel} is stored as a zip archive (apify push does this for sources over 3 MiB), and this tool cannot work on zip-stored versions yet.`; - } - // The API's legacy SOURCE_CODE type and any type added later are not in apify-client's enum. - const { sourceType }: { sourceType: string } = version; - return `${versionLabel} has source type ${sourceType}, which this tool cannot work on; only versions stored as files are supported.`; -} - -/** Throws `UserInputError` for a version not stored as files and for one whose source the API hides. */ -function extractVersionFiles(version: ActorVersion, versionLabel: string): SourceFile[] { - if (version.sourceType !== ActorSourceType.SourceFiles) { - throw new UserInputError(formatSourceTypeRefusal(version, versionLabel)); - } - // The API returns only the number, type, and build tag of a version whose source it hides from this account. - if (!version.sourceFiles) { - throw new UserInputError( - `${versionLabel} came back without its source: the API hides it from accounts that cannot modify the Actor. Ask the Actor's owner for the source.`, - ); - } - return buildFilesManifest(version.sourceFiles); -} - /** Lines with their line endings kept, so joined back they give the exact text. */ function splitLines(text: string): string[] { if (text === '') return []; @@ -264,17 +177,11 @@ export const getActorVersion: ToolEntry = Object.freeze({ } try { // TODO: The Actor GET returns every file of every version; read only the files asked for once the API can. - // apify-client turns username/name into the API's username~name. - const actor = await client.actor(parsed.actor).get(); - // Extra path segments, such as username/name/runs/last, reach a sub-resource that is not an Actor. - if (!actor || typeof actor.name !== 'string' || typeof actor.username !== 'string') { - return respondUserError( - `Actor '${parsed.actor}' not found. Give its ID or its full name, username/name; a name without the username is not enough.`, - ); - } - const fullName = `${actor.username}/${actor.name}`; + const { actor, fullName } = await fetchActor(client, parsed.actor); const version = resolveVersion(actor, parsed.versionNumber, parsed.actor); - const files = extractVersionFiles(version, `Version ${version.versionNumber} of ${fullName}`); + const files = buildFilesManifest( + extractSourceFiles(version, `Version ${version.versionNumber} of ${fullName}`), + ); const structuredContent = { actorId: actor.id, fullName, @@ -299,8 +206,7 @@ export const getActorVersion: ToolEntry = Object.freeze({ .join(' '); return respondOk([JSON.stringify(structuredContent), summary], { structuredContent }); } catch (error) { - if (error instanceof UserInputError) return respondUserError(error.message); - throw error; + return respondToSourceToolError(error); } }, } as const); diff --git a/src/tools/source/source_files.ts b/src/tools/versions/source_files.ts similarity index 86% rename from src/tools/source/source_files.ts rename to src/tools/versions/source_files.ts index 8631349e2..ad5edb1e1 100644 --- a/src/tools/source/source_files.ts +++ b/src/tools/versions/source_files.ts @@ -74,6 +74,8 @@ export type SourceFile = { contentBytes: number; /** The content as returned: the text for utf8, the base64 for base64. */ content: string; + /** The entry as stored, which a write sends back for a file it leaves alone. */ + entry: ActorVersionSourceFile; }; export function hasBinaryExtension(path: string): boolean { @@ -93,14 +95,14 @@ export function compareSourcePaths(a: string, b: string): number { } /** - * Identifies the file set: one `path\0hash\n` line per regular file, sorted by path. The stored format does not go - * in, so a file stored as TEXT or as BASE64 gives the same revision. + * Identifies the file set: the JSON array of `[path, hash]` pairs of its regular files, sorted by path. JSON quotes + * each path and escapes quotes and control characters in it, so two different file sets never give the same text, as + * a stored name with NUL and a line break once did. The stored format does not go in, so a file stored as TEXT or as + * BASE64 gives the same revision. */ export function buildFilesRevision(files: readonly Pick[]): string { - const lines = [...files] - .sort((a, b) => compareSourcePaths(a.path, b.path)) - .map(({ path, hash }) => `${path}\0${hash}\n`); - return getSha256Prefix(lines.join('')); + const pairs = [...files].sort((a, b) => compareSourcePaths(a.path, b.path)).map(({ path, hash }) => [path, hash]); + return getSha256Prefix(JSON.stringify(pairs)); } /** @@ -113,7 +115,7 @@ export function buildInlineSourceFile(file: ActorVersionSourceFile): SourceFile const content = storedContent ?? ''; const path = posix.normalize(name); const bytes = Buffer.from(content, format === 'BASE64' ? 'base64' : 'utf8'); - const common = { path, sizeBytes: bytes.length, hash: getSha256Prefix(bytes) }; + const common = { path, sizeBytes: bytes.length, hash: getSha256Prefix(bytes), entry: file }; if (format !== 'BASE64') { return { ...common, encoding: 'utf8', contentBytes: bytes.length, content }; } diff --git a/src/tools/versions/source_helpers.ts b/src/tools/versions/source_helpers.ts new file mode 100644 index 000000000..8fc708e81 --- /dev/null +++ b/src/tools/versions/source_helpers.ts @@ -0,0 +1,396 @@ +import { posix } from 'node:path'; + +import type { Actor, ActorVersion, ActorVersionSourceFile, Build } from 'apify-client'; +import { ActorSourceType, ApifyApiError } from 'apify-client'; +import { z } from 'zod'; + +import type { ApifyClient } from '../../apify_client.js'; +import { UserInputError } from '../../errors.js'; +import { REDACTED } from '../../resources/api_resources.js'; +import type { ConsoleLinkContext, InternalToolArgs } from '../../types.js'; +import { getResourceByIdOrName } from '../../utils/apify_resource_id.js'; +import { getConsoleLinkContext } from '../../utils/console_link.js'; +import { logHttpError } from '../../utils/logging.js'; +import type { ToolResponse } from '../../utils/mcp.js'; +import { respondAborted, respondUserError } from '../../utils/mcp.js'; +import { isRecord } from '../api/apify_api_spec.js'; +import { buildNextStepForBuild, listVersionNumbers, respondWithBuild, toBuildResult } from '../builds/build_helpers.js'; +import { hasBinaryExtension } from './source_files.js'; + +/** The input shape of one file sent whole. */ +export const sourceFileArgs = z.object({ + path: z.string().min(1).describe('Path relative to the Actor root, for example src/main.js.'), + content: z.string().describe('The whole file content: text as is, or base64 when encoding is base64.'), + encoding: z + .enum(['utf8', 'base64']) + .optional() + .describe( + 'utf8 for text, base64 for binary files. Defaults to base64 for binary extensions such as .png and to utf8 otherwise.', + ), +}); + +/** `keys` of `record` that are set: a string, number, or boolean as is, any other value as `REDACTED`. */ +function pickLoggedValues(record: Record, keys: readonly string[]): Record { + return Object.fromEntries( + keys.flatMap((key) => { + const value = record[key]; + if (value === undefined) return []; + return [[key, ['string', 'number', 'boolean'].includes(typeof value) ? value : REDACTED]]; + }), + ); +} + +/** `keys` of `record` that are set, each as `REDACTED`. */ +function maskLoggedValues(record: Record, keys: readonly string[]): Record { + return Object.fromEntries(keys.filter((key) => record[key] !== undefined).map((key) => [key, REDACTED])); +} + +/** The logged copy of an entry's edits, each with its oldText and newText as `REDACTED`. */ +function redactEdits(edits: unknown): unknown { + if (!Array.isArray(edits)) return REDACTED; + return edits.map((textEdit) => + isRecord(textEdit) ? maskLoggedValues(textEdit, ['oldText', 'newText']) : REDACTED, + ); +} + +/** + * The logged copy of a source tool's arguments (`redactArgs`), built as an allowlist: `keys`, and each entry of the + * list under `listKey` with its `entryKeys`, and with its content and its edits' oldText and newText as `REDACTED`. + * File content can hold secrets, such as an API key written into the code. + */ +export function redactSourceToolArgs( + args: Record, + fields: { keys: readonly string[]; listKey: string; entryKeys: readonly string[] }, +): Record { + const { keys, listKey, entryKeys } = fields; + const redactEntry = (entry: unknown) => { + if (!isRecord(entry)) return REDACTED; + const { edits } = entry; + return { + ...pickLoggedValues(entry, entryKeys), + ...maskLoggedValues(entry, ['content']), + ...(edits !== undefined && { edits: redactEdits(edits) }), + }; + }; + const list = args[listKey]; + return { + ...pickLoggedValues(args, keys), + ...(list !== undefined && { [listKey]: Array.isArray(list) ? list.map(redactEntry) : REDACTED }), + }; +} + +/** + * The Actor by its ID, its name (resolved against the token's own Actors), or its full name, `username/name` or + * `username~name` (apify-client turns username/name into the API's username~name); throws `UserInputError` when there + * is none. + */ +export async function fetchActor( + client: ApifyClient, + actorSelector: string, +): Promise<{ actor: Actor; fullName: string }> { + const actor = await getResourceByIdOrName(actorSelector, async (safeId) => client.actor(safeId).get()); + // Extra path segments, such as username/name/runs/last, reach a sub-resource that is not an Actor. + if (!actor || typeof actor.name !== 'string' || typeof actor.username !== 'string') { + throw new UserInputError( + `Actor '${actorSelector}' not found. The actor argument takes an ID, a name (resolved against your own Actors), or a full name as username/name.`, + ); + } + return { actor, fullName: `${actor.username}/${actor.name}` }; +} + +/** The requested version, or the only one when none is requested; throws `UserInputError` otherwise. */ +export function resolveVersion( + actor: Pick, + requestedVersionNumber: string | undefined, + actorSelector: string, +): ActorVersion & { versionNumber: string } { + const versionNumbers = listVersionNumbers(actor); + // AJV turns a null versionNumber into an empty string, which no version has, so it counts as not given. + const requested = requestedVersionNumber || undefined; + if (requested === undefined && versionNumbers.length !== 1) { + // The source type and build tag tell the caller which version holds the code it is after. + const versions = actor.versions + .filter((version) => version.versionNumber !== undefined) + .map(({ versionNumber, sourceType, buildTag }) => { + const tag = buildTag ? `, build tag ${buildTag}` : ''; + return `${versionNumber} (${sourceType}${tag})`; + }); + throw new UserInputError(`Specify versionNumber; this Actor has versions: ${versions.join(', ')}.`); + } + const versionNumber = requested ?? versionNumbers[0]; + const version = actor.versions.find((candidate) => candidate.versionNumber === versionNumber); + if (!version) { + throw new UserInputError( + `Actor '${actorSelector}' has no version ${versionNumber}; available versions: ${versionNumbers.join(', ')}.`, + ); + } + return { ...version, versionNumber }; +} + +/** + * The URL without what can grant access to it: the query string, the password, and, for http and https, the user. An + * SSH user such as `git@` is not a secret and stays. A URL the parser cannot read, such as + * `git@github.com:user/repo.git`, loses only its query string. + */ +function formatUrlWithoutSecrets(url: string): string { + if (URL.canParse(url)) { + const parsed = new URL(url); + parsed.search = ''; + parsed.password = ''; + if (parsed.protocol === 'http:' || parsed.protocol === 'https:') { + parsed.username = ''; + } + return parsed.href; + } + const queryIndex = url.indexOf('?'); + if (queryIndex === -1) return url; + const hashIndex = url.indexOf('#', queryIndex); + return url.slice(0, queryIndex) + (hashIndex === -1 ? '' : url.slice(hashIndex)); +} + +/** + * Why a version not stored as files is refused, and what to use instead. A version keeps the URL fields of a source + * type it used before, so only the URL of its current type is named. + */ +function formatSourceTypeRefusal(version: ActorVersion, versionLabel: string): string { + if (version.sourceType === ActorSourceType.GitRepo) { + const url = version.gitRepoUrl ? ` ${formatUrlWithoutSecrets(version.gitRepoUrl)}` : ''; + return `${versionLabel} has its files in the Git repository${url}, not stored on Apify, so this tool cannot work on them; use the repository.`; + } + if (version.sourceType === ActorSourceType.GitHubGist) { + const url = version.gitHubGistUrl ? ` ${formatUrlWithoutSecrets(version.gitHubGistUrl)}` : ''; + return `${versionLabel} has its files in the GitHub gist${url}, not stored on Apify, so this tool cannot work on them; use the gist.`; + } + // TODO(#1452): Read and write zip-stored (TARBALL) versions with adm-zip 0.6.1 or later, within the download and + // unpacking limits the issue lists. + if (version.sourceType === ActorSourceType.Tarball) { + return `${versionLabel} is stored as a zip archive (apify push does this for sources over 3 MiB), and this tool cannot work on zip-stored versions yet.`; + } + // The API's legacy SOURCE_CODE type and any type added later are not in apify-client's enum. + const { sourceType }: { sourceType: string } = version; + return `${versionLabel} has source type ${sourceType}, which this tool cannot work on; only versions stored as files are supported.`; +} + +/** + * The version's stored entries, folders included. Throws `UserInputError` for a version not stored as files and for + * one whose source the API hides. + */ +export function extractSourceFiles(version: ActorVersion, versionLabel: string): ActorVersionSourceFile[] { + if (version.sourceType !== ActorSourceType.SourceFiles) { + throw new UserInputError(formatSourceTypeRefusal(version, versionLabel)); + } + // The API returns only the number, type, and build tag of a version whose source it hides from this account. + if (!version.sourceFiles) { + throw new UserInputError( + `${versionLabel} came back without its source: the API hides it from accounts that cannot modify the Actor. Ask the Actor's owner for the source.`, + ); + } + return version.sourceFiles; +} + +/** + * The entry to store for a file sent whole: TEXT for utf8, BASE64 for base64, where no encoding means base64 for a + * binary extension, the rule get-actor-version reads by. The path is normalized the way get-actor-version lists it. + */ +export function buildSourceFileEntry({ + path, + content, + encoding, +}: z.infer): ActorVersionSourceFile { + const isBase64 = (encoding ?? (hasBinaryExtension(path) ? 'base64' : 'utf8')) === 'base64'; + return { name: posix.normalize(path), format: isBase64 ? 'BASE64' : 'TEXT', content }; +} + +/** + * Base64 that decodes to its bytes without loss: Node and the build worker skip characters they cannot decode and stop + * at padding, so text sent as base64 would be stored as other bytes. Line breaks, missing padding, and the URL-safe + * alphabet decode without loss. + */ +function isBase64(content: string): boolean { + const compact = content.replace(/\s/g, '').replace(/-/g, '+').replace(/_/g, '/'); + return Buffer.from(compact, 'base64').toString('base64').replace(/=+$/, '') === compact.replace(/=+$/, ''); +} + +/** + * Throws `UserInputError` for content the stored file would not hold as sent: base64 that does not decode without + * loss, and text with a lone UTF-16 surrogate, which UTF-8 stores as U+FFFD. + */ +export function validateFileContent({ format, content }: ActorVersionSourceFile, label: string): void { + if (format === 'BASE64' && !isBase64(content)) { + throw new UserInputError( + `${label} has content that is not valid base64; encoding base64 takes binary content as base64, and text takes encoding utf8.`, + ); + } + if (format === 'TEXT' && !content.isWellFormed()) { + throw new UserInputError(`${label} has text with a lone UTF-16 surrogate, which UTF-8 cannot store.`); + } +} + +/** + * Throws `UserInputError` for a path with a control character, U+0000 to U+001F or U+007F, or with a lone UTF-16 + * surrogate. The build writes the files to disk, where a file name cannot hold NUL, and UTF-8 stores a lone surrogate + * as U+FFFD. + */ +export function validatePathCharacters(path: string, label: string): void { + if (!path.isWellFormed()) { + throw new UserInputError(`${label} has a lone UTF-16 surrogate in its path, which UTF-8 cannot store.`); + } + const character = [...path].find((char) => char <= '\u001f' || char === '\u007f'); + if (character === undefined) return; + const codePoint = `U+${character.charCodeAt(0).toString(16).toUpperCase().padStart(4, '0')}`; + throw new UserInputError( + `${label} has the control character ${codePoint} in its path; a path cannot hold characters U+0000 to U+001F or U+007F.`, + ); +} + +/** + * Throws `UserInputError` for a normalized path no file can be written at. The build worker refuses a version with a + * path outside the Actor root, so every build of it would fail; `.` and a path ending in a slash name a folder. A + * backslash is refused too: the build takes it as part of the file name, so src\main.js would land in the Actor root. + */ +export function validateFilePath(path: string, label: string): void { + if (path.includes('\\')) { + throw new UserInputError(`${label} has a backslash in its path; paths use / as the separator.`); + } + if (path.startsWith('/') || path === '..' || path.startsWith('../')) { + throw new UserInputError( + `${label} has a path outside the Actor root; paths are relative to it, such as src/main.js.`, + ); + } + if (path === '.' || path.endsWith('/')) { + throw new UserInputError(`${label} has a path that names a folder, not a file.`); + } +} + +/** The folders above a file, outermost first: `a` and `a/b` for `a/b/c.js`. */ +function listFolders(path: string): string[] { + const folders: string[] = []; + for (let slashIndex = path.indexOf('/'); slashIndex !== -1; slashIndex = path.indexOf('/', slashIndex + 1)) { + folders.push(path.slice(0, slashIndex)); + } + return folders; +} + +/** + * The paths of a version's files and folder entries, to check where a new file can go: one path cannot be both a file + * and a folder, and the build worker fails to write such a file. Each check and change looks up only the folders of + * its path, since comparing each file with every other took 8 s for 20,000 files. + */ +export class SourcePathIndex { + private readonly filePaths = new Set(); + + /** The files under each folder, at any depth, in the order they were added. */ + private readonly filePathsByFolder = new Map>(); + + private readonly folderPaths: readonly string[]; + + constructor(filePaths: Iterable, folderPaths: readonly string[] = []) { + this.folderPaths = folderPaths; + for (const path of filePaths) this.addFile(path); + } + + hasFile(path: string): boolean { + return this.filePaths.has(path); + } + + /** A file added again keeps its place in the order. */ + addFile(path: string): void { + this.filePaths.add(path); + for (const folder of listFolders(path)) { + this.filePathsByFolder.set(folder, (this.filePathsByFolder.get(folder) ?? new Set()).add(path)); + } + } + + deleteFile(path: string): void { + this.filePaths.delete(path); + for (const folder of listFolders(path)) this.filePathsByFolder.get(folder)?.delete(path); + } + + /** + * Throws `UserInputError` when a new file at `path` collides with another file or folder: a file at a folder of + * `path`, or else the first file added under `path`, or else the first folder entry at or under `path`. A file at + * `path` itself is for the caller to check. + */ + validateNewFilePath(path: string, label: string): void { + const collision = + listFolders(path).find((folder) => this.filePaths.has(folder)) ?? + this.filePathsByFolder.get(path)?.values().next().value ?? + this.folderPaths.find((folderPath) => folderPath === path || folderPath.startsWith(`${path}/`)); + if (collision === undefined) return; + throw new UserInputError(`${label} collides with ${collision}; one path cannot be both a file and a folder.`); + } +} + +/** The build worker skips a file whose content is empty, so the caller hears of such files. */ +export function buildEmptyFilesWarnings(paths: readonly string[]): { warnings?: string[] } { + if (paths.length === 0) return {}; + const warning = `These files are empty, and the build skips empty files, so they will not exist in the build: ${paths.join(', ')}.`; + return { warnings: [warning] }; +} + +/** + * Starts a build of the version after a committed write, with no tag so the version's buildTag applies, and does not + * wait for it. The write stands either way, so a failed start is returned as `buildError` rather than thrown: a caller + * told the call failed would retry writes that were saved. A cancel that arrives before the start starts no build. + */ +async function startBuildAfterWrite( + toolArgs: Pick, + target: { actorId: string; versionNumber: string }, +): Promise<{ build: Build; linkContext: ConsoleLinkContext | undefined } | { buildError: string } | { aborted: true }> { + const { apifyClient: client, apifyToken, signal } = toolArgs; + try { + // Resolved before the start, so a failed lookup leaves no build behind. + const linkContext = await getConsoleLinkContext(apifyToken, client); + // Per the MCP spec a cancelled request gets no response; the write stands. + if (signal?.aborted) return { aborted: true }; + // Started without the request's signal: a committed write never aborts the build it started. + const build = await client.actor(target.actorId).build(target.versionNumber, { useCache: true }); + return { build, linkContext }; + } catch (error) { + // The API's refusal is the caller's to read; anything else, such as a network failure, is logged too. + if (!(error instanceof ApifyApiError)) + logHttpError(error, 'Failed to start a build after a source write', target); + return { buildError: error instanceof Error ? error.message : String(error) }; + } +} + +/** + * The response to a committed write. With autoBuild, a build of the version starts first; a cancel that arrives before + * the start gets the cancelled-call response. + */ +export async function respondAfterWrite(params: { + toolArgs: Pick; + autoBuild: boolean; + target: { actorId: string; versionNumber: string }; + structuredContent: Record; + summary: string; +}): Promise { + const { toolArgs, autoBuild, target, structuredContent, summary } = params; + if (!autoBuild) { + return respondWithBuild({ + structuredContent, + summary, + nextStep: 'No build was started; runs use these files only after the version is built.', + }); + } + const started = await startBuildAfterWrite(toolArgs, target); + if ('aborted' in started) return respondAborted(); + if ('buildError' in started) { + return respondWithBuild({ + structuredContent: { ...structuredContent, buildError: started.buildError }, + summary, + nextStep: 'The build could not be started; runs use these files only after the version is built.', + }); + } + return respondWithBuild({ + structuredContent: { ...structuredContent, build: toBuildResult(started.build, started.linkContext) }, + summary, + nextStep: buildNextStepForBuild(started.build, { loadedToolNames: toolArgs.loadedToolNames }), + }); +} + +/** A `UserInputError` as a soft failure; anything else is rethrown for the tool-call engine to report. */ +export function respondToSourceToolError(error: unknown): ToolResponse { + if (error instanceof UserInputError) return respondUserError(error.message); + throw error; +} diff --git a/src/tools/versions/update_actor_version.ts b/src/tools/versions/update_actor_version.ts new file mode 100644 index 000000000..43a99bb5e --- /dev/null +++ b/src/tools/versions/update_actor_version.ts @@ -0,0 +1,521 @@ +import { posix } from 'node:path'; + +import { ActorSourceType, ApifyApiError } from 'apify-client'; +import dedent from 'dedent'; +import { z } from 'zod'; + +import { HELPER_TOOLS } from '../../const.js'; +import { UserInputError } from '../../errors.js'; +import type { InternalToolArgs, ToolEntry, ToolInputSchema } from '../../types.js'; +import { TOOL_TYPE } from '../../types.js'; +import { compileSchema, fixZodSchemaRequired } from '../../utils/ajv.js'; +import { getApifyErrorType } from '../../utils/apify_errors.js'; +import { getHttpStatusCode, logHttpError } from '../../utils/logging.js'; +import type { ToolResponse } from '../../utils/mcp.js'; +import { respondAborted, respondServerError, respondUserError } from '../../utils/mcp.js'; +import { respondWithBuild } from '../builds/build_helpers.js'; +import { updateActorVersionToolOutputSchema } from '../structured_output_schemas.js'; +import type { SourceFile } from './source_files.js'; +import { + buildFilesManifest, + buildFilesRevision, + buildInlineSourceFile, + compareSourcePaths, + isFolderEntry, +} from './source_files.js'; +import { + buildEmptyFilesWarnings, + buildSourceFileEntry, + extractSourceFiles, + fetchActor, + redactSourceToolArgs, + resolveVersion, + respondAfterWrite, + respondToSourceToolError, + SourcePathIndex, + validateFileContent, + validateFilePath, + validatePathCharacters, +} from './source_helpers.js'; + +/** Why an operation or the revision check failed; the caller reads the code to decide how to recover. */ +const PRECONDITION_REASON = { + FILE_EXISTS: 'FILE_EXISTS', + FILE_NOT_FOUND: 'FILE_NOT_FOUND', + HASH_MISMATCH: 'HASH_MISMATCH', + NO_MATCH: 'NO_MATCH', + MULTIPLE_MATCHES: 'MULTIPLE_MATCHES', + NOT_TEXT: 'NOT_TEXT', + REVISION_MISMATCH: 'REVISION_MISMATCH', +} as const; +type PRECONDITION_REASON = (typeof PRECONDITION_REASON)[keyof typeof PRECONDITION_REASON]; + +// Each edit searches the whole text of its file, which is at most MAX_EDITED_TEXT_LENGTH long, and blocks the event loop +// meanwhile: up to about 20 ms for an oldText over 250 characters, so 20 edits take up to about 0.4 s. Writes and deletes +// never search a stored file. +const MAX_OPERATIONS_PER_CALL = 100; +const MAX_EDITS_PER_CALL = 20; +// In UTF-16 code units, which a string's length gives at no cost and the search time grows with. The platform refuses +// a version whose stored file contents pass 3 MiB in total, in UTF-8 bytes (MAX_MULTIFILE_BYTES in @apify/consts), but +// checks that only at the PUT, after the edits ran. UTF-8 takes at least one byte per code unit, so no file the platform +// stored is longer than this; a file within it can still be over the platform's limit, which the PUT then refuses. +const MAX_EDITED_TEXT_LENGTH = 3 * 1024 * 1024; + +// One flat object with an explicit `type`: AJV's `removeAdditional` drops the fields of all but one branch of a +// union, so the field each type needs is checked in code. +const operationArgs = z.object({ + type: z + .enum(['write', 'edit', 'delete']) + .describe('write creates or replaces a file, edit replaces text in one, delete removes one.'), + path: z.string().min(1).describe('The file, relative to the Actor root, for example src/main.js.'), + content: z.string().optional().describe('write: the whole new content.'), + encoding: z + .enum(['utf8', 'base64']) + .optional() + .describe( + 'write: utf8 for text, base64 for binary files. Defaults to base64 for binary extensions such as .png and to utf8 otherwise.', + ), + expectedHash: z + .string() + .optional() + .describe( + "write over an existing file, and delete: the file's hash from the version listing. Leave it out to create a new file. An edit checks it when given.", + ), + edits: z + .array( + z.object({ + oldText: z.string().min(1).describe('The exact current text to replace, copied from the file content.'), + newText: z.string().describe('The text to put in its place; empty to remove oldText.'), + }), + ) + .min(1) + .optional() + .describe('edit: replacements applied in order, each to the text the previous ones left.'), +}); + +type OperationArgs = z.infer; + +type TextEdit = NonNullable[number]; + +const updateActorVersionArgs = z.object({ + actor: z + .string() + .min(1) + .describe( + 'The Actor to change: its ID, its name (resolved against your own Actors), or its full name as username/name or username~name.', + ), + versionNumber: z + .string() + .optional() + .describe( + 'Version to change in MAJOR.MINOR form, for example 0.1. Defaults to the only version when the Actor has exactly one.', + ), + operations: z + .array(operationArgs) + .max(MAX_OPERATIONS_PER_CALL) + .describe( + 'File changes, applied in order to the current files. They are saved together, or none is saved when any fails.', + ), + expectedRevision: z + .string() + .optional() + .describe( + 'The revision of the version when you read it; the call fails if any file changed since. ' + + 'Pass it so that a retried call fails instead of applying its edits twice.', + ), + autoBuild: z + .boolean() + .default(false) + .describe('Start a build of the version after the write, and return without waiting for it. Default: false.'), +}); + +type FileChange = { path: string; action: 'created' | 'updated' | 'deleted'; hash?: string }; + +/** `label` names the operation, or expectedRevision. */ +function buildPreconditionError(label: string, reason: PRECONDITION_REASON, detail: string): UserInputError { + return new UserInputError(`Nothing was written: ${label} failed with ${reason}. ${detail}`); +} + +/** Throws FILE_NOT_FOUND when there is no file at `path`. */ +function findFile(files: ReadonlyMap, path: string, label: string): SourceFile { + const file = files.get(path); + if (!file) throw buildPreconditionError(label, PRECONDITION_REASON.FILE_NOT_FOUND, `There is no file at ${path}.`); + return file; +} + +function validateHash(file: SourceFile, expectedHash: string, label: string): void { + if (expectedHash.toLowerCase() === file.hash) return; + const detail = `${file.path} has hash ${file.hash}, not ${expectedHash}.`; + throw buildPreconditionError(label, PRECONDITION_REASON.HASH_MISMATCH, detail); +} + +/** Every line break is CRLF; a file with mixed line endings gets no conversion. */ +function hasOnlyCrlfLineBreaks(text: string): boolean { + return text.includes('\r\n') && !/(? 0 && pattern.charCodeAt(i) !== pattern.charCodeAt(matched)) matched = fallback[matched - 1]; + if (pattern.charCodeAt(i) === pattern.charCodeAt(matched)) matched++; + fallback[i] = matched; + } + let firstOffset: number | undefined; + for (let i = 0, matched = 0; i < text.length; i++) { + const charCode = text.charCodeAt(i); + while (matched > 0 && charCode !== pattern.charCodeAt(matched)) matched = fallback[matched - 1]; + if (charCode === pattern.charCodeAt(matched)) matched++; + if (matched === pattern.length) { + const offset = i - matched + 1; + if (firstOffset !== undefined) return [firstOffset, offset]; + firstOffset = offset; + matched = fallback[matched - 1]; + } + } + return firstOffset === undefined ? [] : [firstOffset]; +} + +/** + * The text after the edits, each applied to the text the previous ones left. Each oldText must match exactly once, + * byte for byte. The one exception: in a file with only CRLF line breaks, LF in oldText and newText is taken as CRLF, + * since models write LF. Matched as given, such an LF could split a CRLF, and inserted, it would mix line endings. + */ +function applyTextEdits(originalText: string, edits: readonly TextEdit[], label: string): string { + const isCrlfFile = hasOnlyCrlfLineBreaks(originalText); + let text = originalText; + for (const [editIndex, edit] of edits.entries()) { + // A lone surrogate can match half of a character, and UTF-8 stores it as U+FFFD. + if (!edit.oldText.isWellFormed() || !edit.newText.isWellFormed()) { + throw new UserInputError( + `${label} has a lone UTF-16 surrogate in edits[${editIndex}], which UTF-8 cannot store.`, + ); + } + // An upper bound of the length after this edit, taken before the CRLF conversion and the search: converting only + // lengthens oldText, and lengthens newText by one code unit for each LF it turns into CRLF. + const newTextLength = edit.newText.length + (isCrlfFile ? countLoneLineFeeds(edit.newText) : 0); + if (text.length - edit.oldText.length + newTextLength > MAX_EDITED_TEXT_LENGTH) { + throw new UserInputError( + `Nothing was written: edits[${editIndex}] of ${label} can make the file longer than ${MAX_EDITED_TEXT_LENGTH} UTF-16 code units, the limit for an edited file.`, + ); + } + const oldText = isCrlfFile ? convertToCrlf(edit.oldText) : edit.oldText; + const newText = isCrlfFile ? convertToCrlf(edit.newText) : edit.newText; + const [offset, nextOffset] = findMatchOffsets(text, oldText); + if (offset === undefined) { + const detail = `oldText of edits[${editIndex}] is not in the file.`; + throw buildPreconditionError(label, PRECONDITION_REASON.NO_MATCH, detail); + } + if (nextOffset !== undefined) { + const detail = `oldText of edits[${editIndex}] matches more than once; surrounding lines in oldText can narrow it to one match.`; + throw buildPreconditionError(label, PRECONDITION_REASON.MULTIPLE_MATCHES, detail); + } + text = text.slice(0, offset) + newText + text.slice(offset + oldText.length); + } + return text; +} + +/** The file an operation names, with the expectedHash it gives and the label its errors start with. */ +type OperationTarget = { path: string; expectedHash: string | undefined; label: string }; + +/** Writes the file at `target.path` into `version.files`; throws `UserInputError` when it cannot. */ +function applyWrite( + version: { files: Map; pathIndex: SourcePathIndex }, + target: OperationTarget, + { content, encoding }: Pick, +): void { + const { files, pathIndex } = version; + const { path, expectedHash, label } = target; + if (content === undefined) throw new UserInputError(`${label} needs content.`); + // Only a write is checked, so a stored file at such a path can still be edited or deleted. + validateFilePath(path, label); + const existing = files.get(path); + if (existing && expectedHash === undefined) { + const detail = `${path} exists with hash ${existing.hash}; a write with that hash as expectedHash replaces it.`; + throw buildPreconditionError(label, PRECONDITION_REASON.FILE_EXISTS, detail); + } + if (expectedHash !== undefined) validateHash(findFile(files, path, label), expectedHash, label); + if (!existing) pathIndex.validateNewFilePath(path, label); + const entry = buildSourceFileEntry({ path, content, encoding }); + validateFileContent(entry, label); + files.set(path, buildInlineSourceFile(entry)); + pathIndex.addFile(path); +} + +/** Applies `edits` to the file at `target.path` in `files`; throws `UserInputError` when they cannot apply. */ +function applyEdit(files: Map, target: OperationTarget, edits: OperationArgs['edits']): void { + const { path, expectedHash, label } = target; + if (edits === undefined) throw new UserInputError(`${label} needs edits.`); + const existing = findFile(files, path, label); + if (expectedHash !== undefined) validateHash(existing, expectedHash, label); + // The same rule get-actor-version returns text by, so a file it returned as utf8 can be edited. + if (existing.encoding !== 'utf8') { + const detail = `${path} is not UTF-8 text; only a write can replace it.`; + throw buildPreconditionError(label, PRECONDITION_REASON.NOT_TEXT, detail); + } + const text = applyTextEdits(existing.content, edits, label); + // A UTF-8 file that `apify push` stored as BASE64 stays BASE64. + const isBase64 = existing.entry.format === 'BASE64'; + const newContent = isBase64 ? Buffer.from(text).toString('base64') : text; + files.set(path, buildInlineSourceFile({ name: path, format: isBase64 ? 'BASE64' : 'TEXT', content: newContent })); +} + +/** Deletes the file at `target.path` from `version.files`; throws `UserInputError` when it cannot. */ +function applyDelete( + version: { files: Map; pathIndex: SourcePathIndex }, + target: OperationTarget, +): void { + const { files, pathIndex } = version; + const { path, expectedHash, label } = target; + if (expectedHash === undefined) throw new UserInputError(`${label} needs expectedHash.`); + validateHash(findFile(files, path, label), expectedHash, label); + files.delete(path); + pathIndex.deleteFile(path); +} + +/** + * The files after the operations, each applied to the files the previous ones left; throws `UserInputError` for the + * first operation that cannot apply, before anything is written. + */ +function applyOperations( + version: { files: ReadonlyMap; folderPaths: readonly string[] }, + operations: readonly OperationArgs[], +): Map { + const files = new Map(version.files); + const pathIndex = new SourcePathIndex(files.keys(), version.folderPaths); + for (const [index, operation] of operations.entries()) { + const { type } = operation; + const target = { + // Normalized the way get-actor-version lists paths, so a listed path always matches. + path: posix.normalize(operation.path), + // AJV turns a null expectedHash into an empty string, which no file has, so it counts as not given. + expectedHash: operation.expectedHash || undefined, + label: `operations[${index}] (${type} ${operation.path})`, + }; + if (type === 'write') applyWrite({ files, pathIndex }, target, operation); + if (type === 'edit') applyEdit(files, target, operation.edits); + if (type === 'delete') applyDelete({ files, pathIndex }, target); + } + return files; +} + +/** One entry per file the operations created, updated, or deleted, sorted by path. */ +function buildFileChanges( + before: ReadonlyMap, + after: ReadonlyMap, +): FileChange[] { + const paths = [...new Set([...before.keys(), ...after.keys()])].sort(compareSourcePaths); + return paths.flatMap((path): FileChange[] => { + const file = after.get(path); + if (!file) return [{ path, action: 'deleted' }]; + const previousHash = before.get(path)?.hash; + if (previousHash === file.hash) return []; + return [{ path, action: previousHash === undefined ? 'created' : 'updated', hash: file.hash }]; + }); +} + +/** + * The response to a version PUT that failed with a 5xx, without an HTTP response, such as on a timeout, or on an attempt + * after the first, whatever its status: the platform may have saved the files of that attempt, or of an earlier one + * apify-client retried, so the reply gives the revisions that show whether it did. Any other error, such as a 4xx on + * the first attempt, is rethrown. + */ +function respondToUnconfirmedSave( + error: unknown, + params: { + target: { actorId: string; versionNumber: string; fullName: string }; + previousRevision: string; + revision: string; + autoBuild: boolean; + loadedToolNames: readonly string[]; + }, +): ToolResponse { + const { target, previousRevision, revision, autoBuild, loadedToolNames } = params; + const httpStatus = getHttpStatusCode(error); + const isRetried = error instanceof ApifyApiError && error.attempt > 1; + if (!isRetried && httpStatus !== undefined && httpStatus < 500) throw error; + logHttpError(error, 'Failed to save the files of an Actor version', { + actorId: target.actorId, + versionNumber: target.versionNumber, + }); + const message = error instanceof Error ? error.message : String(error); + const apiErrorType = getApifyErrorType(error); + const failure = + httpStatus === undefined + ? `without an HTTP response: ${message}` + : `with HTTP ${httpStatus}: ${message}${apiErrorType ? ` (API error type: ${apiErrorType})` : ''}`; + const readTool = loadedToolNames.includes(HELPER_TOOLS.ACTOR_VERSION_GET) + ? `; ${HELPER_TOOLS.ACTOR_VERSION_GET} reads the revision` + : ''; + // autoBuild starts a build only after a confirmed save, so runs keep the previous build either way. + const buildNote = autoBuild ? ' No build was started.' : ''; + return respondServerError( + `Saving version ${target.versionNumber} of ${target.fullName} failed ${failure}. ` + + 'The platform may still have saved the change. ' + + `If the version's revision is ${revision}, this call saved it, and if it is ${previousRevision}, ` + + `it saved nothing${readTool}.${buildNote}`, + // The status, category, and detail the tool-call engine records for a thrown error. + { error, detail: message.slice(0, 200) }, + ); +} + +/** + * https://docs.apify.com/api/v2/actor-get + * /v2/actors/{actorId} + * https://docs.apify.com/api/v2/actor-version-put + * /v2/actors/{actorId}/versions/{versionNumber} + * + * The operations apply to the version as read in this call, and the result is stored with one version PUT that + * carries only the source keys, never envVars or buildTag. Conflicts are caught by each operation's check and by + * expectedRevision. A save from Apify Console, `apify push`, or another MCP server that lands between this call's read + * and its PUT is not detected: the platform's version PUT takes no precondition, and this stays so until it gets a + * conditional one. + */ +export const updateActorVersion: ToolEntry = Object.freeze({ + type: TOOL_TYPE.INTERNAL, + name: HELPER_TOOLS.ACTOR_VERSION_UPDATE, + title: 'Update Actor version', + description: dedent` + Change files of an Actor version in one call: write, edit, or delete them. Files the call does not mention stay as they are. + The operations apply in order to a fresh read of the version and are saved together; when any of them fails, nothing is saved. + - write {path, content, encoding?}: creates a file, or replaces one given its expectedHash. + - edit {path, edits: [{oldText, newText}], expectedHash?}: replaces text in a UTF-8 file. Each oldText must match exactly once, byte for byte. + - delete {path, expectedHash}: removes a file. To rename a file, delete it and write it at the new path. + A call takes at most ${MAX_OPERATIONS_PER_CALL} operations and at most ${MAX_EDITS_PER_CALL} edits in total across its edit operations. + expectedHash and expectedRevision take the hash and revision from the version listing. A failed check names the operation, the file, and one of FILE_EXISTS, FILE_NOT_FOUND, HASH_MISMATCH, NO_MATCH, MULTIPLE_MATCHES, NOT_TEXT, or REVISION_MISMATCH. + Binary files take base64 content with encoding base64; files with a binary extension such as .png default to it. + Only versions stored as files can be changed. Env vars and the build tag are never changed. + autoBuild starts a build after the write and returns without waiting. Without it, runs keep using the previous build. + + USAGE: + - Use to fix a bug or add a feature in an Actor's code, a few files at a time. + + USAGE EXAMPLES: + - user_input: Fix the typo in src/main.js of my Actor john/my-scraper + - user_input: Add a README to my Actor and rebuild it`, + // `fixZodSchemaRequired` strips `autoBuild` from `required` because it has a default. + inputSchema: fixZodSchemaRequired(z.toJSONSchema(updateActorVersionArgs)) as ToolInputSchema, + outputSchema: updateActorVersionToolOutputSchema, + ajvValidate: compileSchema(z.toJSONSchema(updateActorVersionArgs)), + annotations: { + title: 'Update Actor version', + readOnlyHint: false, + // Writes and deletes overwrite files. + destructiveHint: true, + idempotentHint: false, + // autoBuild starts a build, like build-actor. + openWorldHint: true, + }, + redactArgs: (args: Record) => + redactSourceToolArgs(args, { + keys: ['actor', 'versionNumber', 'expectedRevision', 'autoBuild'], + listKey: 'operations', + entryKeys: ['type', 'path', 'encoding', 'expectedHash'], + }), + call: async (toolArgs: InternalToolArgs) => { + const { args, apifyClient: client, signal, loadedToolNames } = toolArgs; + const parsed = updateActorVersionArgs.parse(args); + const editCount = parsed.operations + .filter(({ type }) => type === 'edit') + .reduce((count, { edits = [] }) => count + edits.length, 0); + if (editCount > MAX_EDITS_PER_CALL) { + return respondUserError( + `Nothing was written: the operations have ${editCount} edits in total, and one call takes at most ${MAX_EDITS_PER_CALL}.`, + ); + } + try { + for (const [index, { path }] of parsed.operations.entries()) { + validatePathCharacters(path, `operations[${index}]`); + } + // TODO: The Apify API has no way to read or change single files of an Actor's source, so even a one-line + // edit reads the whole version here and writes all its files back with the PUT below. That read and write + // is also what leaves a save landing between them undetected. Once the API can apply atomic changes to an + // Actor's source, send only the changed files instead. + const { actor, fullName } = await fetchActor(client, parsed.actor); + const version = resolveVersion(actor, parsed.versionNumber, parsed.actor); + const { versionNumber } = version; + const storedEntries = extractSourceFiles(version, `Version ${versionNumber} of ${fullName}`); + const before = new Map(buildFilesManifest(storedEntries).map((file) => [file.path, file])); + const previousRevision = buildFilesRevision([...before.values()]); + // AJV turns a null expectedRevision into an empty string, which no version has, so it counts as not given. + if (parsed.expectedRevision && parsed.expectedRevision.toLowerCase() !== previousRevision) { + const detail = `The version's revision is ${previousRevision}, not ${parsed.expectedRevision}.`; + throw buildPreconditionError('expectedRevision', PRECONDITION_REASON.REVISION_MISMATCH, detail); + } + const folderPaths = storedEntries.filter(isFolderEntry).map(({ name }) => posix.normalize(name)); + const after = applyOperations({ files: before, folderPaths }, parsed.operations); + const revision = buildFilesRevision([...after.values()]); + // A cancel during the read writes nothing; per the MCP spec the cancelled request gets no response. + if (signal?.aborted) return respondAborted(); + // Nothing to write, so autoBuild starts no build; the stored files may already be in the build runs use. + if (revision === previousRevision) { + return respondWithBuild({ + structuredContent: { revision, changed: false, changes: [] }, + summary: `Nothing changed in version ${versionNumber} of ${fullName}, so nothing was written.`, + nextStep: 'No build was started.', + }); + } + // Console keeps an empty folder as an entry of its own, which goes back as it was. + const sourceFiles = [ + ...storedEntries.filter(isFolderEntry), + ...[...after.values()].map(({ entry }) => entry), + ]; + try { + await client + .actor(actor.id) + .version(versionNumber) + .update({ sourceType: ActorSourceType.SourceFiles, sourceFiles }); + } catch (error) { + return respondToUnconfirmedSave(error, { + target: { actorId: actor.id, versionNumber, fullName }, + previousRevision, + revision, + autoBuild: parsed.autoBuild, + loadedToolNames, + }); + } + const changes = buildFileChanges(before, after); + const emptyPaths = changes.flatMap(({ path }) => (after.get(path)?.sizeBytes === 0 ? [path] : [])); + return await respondAfterWrite({ + toolArgs, + autoBuild: parsed.autoBuild, + target: { actorId: actor.id, versionNumber }, + structuredContent: { revision, changed: true, changes, ...buildEmptyFilesWarnings(emptyPaths) }, + summary: `Updated version ${versionNumber} of ${fullName}.`, + }); + } catch (error) { + return respondToSourceToolError(error); + } + }, +} as const); diff --git a/src/utils/apify_resource_id.ts b/src/utils/apify_resource_id.ts new file mode 100644 index 000000000..63cfc5be3 --- /dev/null +++ b/src/utils/apify_resource_id.ts @@ -0,0 +1,36 @@ +/** Apify resource IDs are exactly 17 alphanumeric characters. */ +const APIFY_ID_REGEX = /^[a-zA-Z0-9]{17}$/; + +/** + * The API reads an unqualified id as an ID, so a bare resource name has to be prefixed with `~` to be + * resolved against the authenticated user's own resources. Ids that already carry a username, in either + * the `username/name` or `username~name` format, and ids that are already IDs, are returned unchanged. + * + * Applies to every Apify resource the API addresses by ID or name (Actors, tasks, schedules, ...), + * which the API resolves the same way. + */ +export function toSafeResourceId(idOrName: string): string { + const trimmed = idOrName.trim(); + if (trimmed.includes('/') || trimmed.includes('~')) return trimmed; + return APIFY_ID_REGEX.test(trimmed) ? trimmed : `~${trimmed}`; +} + +/** A bare 17-alnum value is shape-identical to an ID and to a legal resource name. */ +export function isAmbiguousResourceId(idOrName: string): boolean { + const trimmed = idOrName.trim(); + return !trimmed.includes('/') && !trimmed.includes('~') && APIFY_ID_REGEX.test(trimmed); +} + +/** + * Fetches a resource by ID, name, or `username/name` through `fetch`, which receives the API-safe id. + * An ambiguous value cannot be resolved by inspection, so it is read as an ID first and, on a miss, + * retried as the caller's own resource name. + */ +export async function getResourceByIdOrName( + idOrName: string, + fetch: (safeId: string) => Promise, +): Promise { + const resource = await fetch(toSafeResourceId(idOrName)); + if (resource || !isAmbiguousResourceId(idOrName)) return resource; + return fetch(`~${idOrName.trim()}`); +} diff --git a/src/utils/tools_loader.ts b/src/utils/tools_loader.ts index 3d42b0ee1..28a40ccd8 100644 --- a/src/utils/tools_loader.ts +++ b/src/utils/tools_loader.ts @@ -105,7 +105,8 @@ export function isReportProblemExplicitlySelected(input: Input): boolean { * the *internal* tool variants around it differ by mode. * * Selectors classified as "actor names": - * - NOT a retired selector (`RETIRED_SELECTOR_NAMES`: `'preview'`, `'experimental'`, `'add-actor'`, `'get-actor-log'`) + * - NOT a retired selector (`RETIRED_SELECTOR_NAMES`: `'preview'`, `'experimental'`, `'add-actor'`, `'get-actor-log'`, + * `'source'`) * - NOT a category name (from `CATEGORY_NAME_SET`) * - NOT the name of an internal tool in any mode (from `ALL_INTERNAL_TOOL_NAMES`) * @@ -224,7 +225,7 @@ export function getToolsForServerMode( const internalSelections: ToolEntry[] = []; if (selectors !== undefined && selectors.length > 0) { for (const sel of selectors) { - // Retired selectors (add-actor, experimental, preview, get-actor-log) are inert. + // Retired selectors (add-actor, experimental, preview, get-actor-log, source) are inert. if (RETIRED_SELECTOR_NAMES.has(sel)) continue; const categoryTools = categories[sel as ToolCategory]; diff --git a/tests/integration/suite.ts b/tests/integration/suite.ts index 4b362160c..3c4c4fc9c 100644 --- a/tests/integration/suite.ts +++ b/tests/integration/suite.ts @@ -11,6 +11,7 @@ import { storageCases, tasksCases, toolsCases, + versionsCases, } from '../test_kit/index.js'; import type { SuiteClientOptions } from '../test_kit/index.js'; import type { CaseCtx, SuiteClient, Transport } from '../test_kit/types.js'; @@ -58,6 +59,7 @@ export function createIntegrationTestsSuite(options: IntegrationTestsSuiteOption registerCases('tasks', tasksCases, ctx); registerCases('storage', storageCases, ctx); registerCases('schedules', schedulesCases, ctx); + registerCases('versions', versionsCases, ctx); registerCases('payments', paymentsCases, ctx); }, ); diff --git a/tests/test_kit/cases/versions.cases.ts b/tests/test_kit/cases/versions.cases.ts new file mode 100644 index 000000000..fd558839f --- /dev/null +++ b/tests/test_kit/cases/versions.cases.ts @@ -0,0 +1,614 @@ +import { createHash } from 'node:crypto'; + +import type { ActorVersionSourceFile } from 'apify-client'; +import { ActorSourceType } from 'apify-client'; +import { expect } from 'vitest'; + +import type { ApifyClient } from '@apify/actors-mcp-server/internals.js'; + +import { validateStructuredOutputForTool, withClient } from '../helpers.js'; +import type { Case, SuiteClient } from '../types.js'; + +/** + * create-actor and update-actor-version against the live API. Each case reads what the platform stored with the raw + * client, so it checks the stored source, not only what the tool reported. + * + * Names are `test-source--`, never `eval-`, so the eval harness's `eval-*` sweep on the same account + * never deletes an Actor mid-case. No case sets autoBuild, since builds cost time and money, and the build of the one + * build-actor call is aborted in `finally`. Every case deletes its Actor by name in `finally`, so an Actor a failed + * call left behind goes too; apify-client swallows the 404 when there is none. + * + * Tool names are hardcoded, not read from `HELPER_TOOLS`, so a rename fails these tests (CONTRIBUTING.md). + */ + +/** A 1x1 PNG. */ +const PNG_BASE64 = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=='; + +/** Bytes that are not UTF-8, so the tools return them as base64 whatever the path. */ +const NON_UTF8_BYTES = Buffer.from([0x00, 0xff, 0x80, 0x0a]); + +/** Well-formed as a hash or a revision, and matching none in these cases. */ +const WRONG_HASH = '0000000000000000'; + +type ToolCallResult = { isError?: boolean; content?: { type: string; text?: string }[]; structuredContent?: unknown }; + +type FileListingInfo = { path: string; sizeBytes: number; hash: string }; + +type GetActorVersionResult = { + actorId: string; + fullName: string; + versionNumber: string; + revision: string; + files: FileListingInfo[]; + contents: { path: string; content: string; encoding: string }[]; +}; + +type CreateActorResult = Omit; + +type BuildActorResult = { build: { id: string; actorId: string } }; + +type UpdateActorVersionResult = { + revision: string; + changed: boolean; + changes: { path: string; action: string; hash?: string }[]; +}; + +/** A stored entry: a file, or a `{ name, folder: true }` folder that Console keeps, which apify-client's type leaves out. */ +type StoredSourceFile = ActorVersionSourceFile & { folder?: boolean }; + +/** A version as the API returns it to the owner, with every field it has. */ +type StoredVersionInfo = Record & { versionNumber: string; sourceFiles?: StoredSourceFile[] }; + +type StoredActorInfo = { + id: string; + username: string; + name: string; + title?: string; + description?: string; + isPublic: boolean; + modifiedAt: Date; + versions: StoredVersionInfo[]; +}; + +/** Unique per call: three transport dimensions register each case, and CI runs PRs concurrently in one account. */ +function buildUniqueActorName(purpose: string): string { + return `test-source-${Math.random().toString(36).slice(2, 8)}-${purpose}`; +} + +/** + * The `username~name` selector of the Actor this account gets under `name`, resolved before the Actor exists, so the + * cleanup works even when the call that creates the Actor fails after the platform stored it. + */ +async function fetchOwnActorSelector(api: ApifyClient, name: string): Promise { + const { username } = await api.user().get(); + return `${username}~${name}`; +} + +/** The hash the source tools list: the first 16 hex characters of the SHA-256 of the bytes. */ +function getFileHash(data: Buffer | string): string { + return createHash('sha256').update(data).digest('hex').slice(0, 16); +} + +/** The listing entry get-actor-version gives a file with these bytes; a string counts as its UTF-8 bytes. */ +function buildFileListing(path: string, data: Buffer | string): FileListingInfo { + const bytes = Buffer.from(data); + return { path, sizeBytes: bytes.length, hash: getFileHash(bytes) }; +} + +async function callTool(client: SuiteClient, name: string, args: Record): Promise { + return (await client.callTool({ name, arguments: args })) as ToolCallResult; +} + +/** + * Fails the case on a tool error, checks the result against the tool's output schema and against its JSON text, which + * a client that reads only the text gets, and returns the structured content. + */ +function expectToolSuccess(result: ToolCallResult, toolName: string): T { + expect(result.isError, JSON.stringify(result.content)).not.toBe(true); + validateStructuredOutputForTool(result, toolName, 'default'); + expect(JSON.parse(result.content?.[0]?.text ?? 'null')).toEqual(result.structuredContent); + return result.structuredContent as T; +} + +/** The one text of a failed call, which carries no structured content. */ +function expectToolFailure(result: ToolCallResult): string { + expect(result.isError).toBe(true); + expect(result.structuredContent).toBeUndefined(); + expect(result.content).toHaveLength(1); + return result.content?.[0]?.text ?? ''; +} + +async function fetchStoredActor(api: ApifyClient, actorSelector: string): Promise { + const actor = await api.actor(actorSelector).get(); + expect(actor, `Actor ${actorSelector}`).toBeDefined(); + return actor as unknown as StoredActorInfo; +} + +/** What a call that writes nothing leaves as it was: the versions, and modifiedAt, which every version write sets. */ +function extractWrittenState({ modifiedAt, versions }: StoredActorInfo) { + return { modifiedAt, versions }; +} + +/** The stored entries sorted by name; the order the platform keeps them in is not part of the contract. */ +function extractSortedSourceFiles(version: StoredVersionInfo): StoredSourceFile[] { + return [...(version.sourceFiles ?? [])].sort((a, b) => (a.name < b.name ? -1 : 1)); +} + +async function fetchBuildCount(api: ApifyClient, actorId: string): Promise { + const { total } = await api.actor(actorId).builds().list(); + return total; +} + +export const versionsCases: Case[] = [ + { + name: 'create-actor creates a private Actor whose stored files get-actor-version reads back with the same hashes and revision', + isDeploymentTest: false, + run: withClient({ tools: ['versions', 'build-actor'] }, async (client, ctx) => { + const api = ctx.createApifyClient(); + const name = buildUniqueActorName('create'); + const selector = await fetchOwnActorSelector(api, name); + // No name, which the platform adds, so the stored file differs from the one sent. + const actorJson = '{\n "actorSpecification": 1,\n "version": "0.0"\n}\n'; + const mainJs = 'import { greet } from "./lib/greet.js";\n\nconsole.log(greet("Grüße 🌍"));\n'; + const greetJs = 'export const greet = (who) => "Hello, " + who;\n'; + let buildId: string | undefined; + try { + const createResult = await callTool(client, 'create-actor', { + name, + title: 'Source tools test', + description: 'Created by an integration test of create-actor.', + files: [ + { path: '.actor/actor.json', content: actorJson }, + { path: 'src/main.js', content: mainJs }, + { path: 'src/lib/greet.js', content: greetJs }, + // Binary by its extension, so base64 without an encoding. + { path: 'assets/logo.png', content: PNG_BASE64 }, + { path: 'assets/raw-bytes', content: NON_UTF8_BYTES.toString('base64'), encoding: 'base64' }, + ], + }); + const created = expectToolSuccess(createResult, 'create-actor'); + const stored = await fetchStoredActor(api, selector); + const fullName = `${stored.username}/${name}`; + + expect(stored).toMatchObject({ + id: created.actorId, + name, + title: 'Source tools test', + description: 'Created by an integration test of create-actor.', + isPublic: false, + }); + expect(stored.versions).toHaveLength(1); + expect(stored.versions[0]).toMatchObject({ + versionNumber: '0.0', + buildTag: 'latest', + sourceType: 'SOURCE_FILES', + envVars: [], + }); + + const storedFiles = extractSortedSourceFiles(stored.versions[0]); + const storedActorJson = storedFiles[0].content; + expect(JSON.parse(storedActorJson)).toEqual({ actorSpecification: 1, version: '0.0', name }); + expect(storedFiles).toEqual([ + { name: '.actor/actor.json', format: 'TEXT', content: storedActorJson }, + { name: 'assets/logo.png', format: 'BASE64', content: PNG_BASE64 }, + { name: 'assets/raw-bytes', format: 'BASE64', content: NON_UTF8_BYTES.toString('base64') }, + { name: 'src/lib/greet.js', format: 'TEXT', content: greetJs }, + { name: 'src/main.js', format: 'TEXT', content: mainJs }, + ]); + + // The listing has the stored bytes of actor.json, not the bytes sent. + const expectedFiles = [ + buildFileListing('.actor/actor.json', storedActorJson), + buildFileListing('assets/logo.png', Buffer.from(PNG_BASE64, 'base64')), + buildFileListing('assets/raw-bytes', NON_UTF8_BYTES), + buildFileListing('src/lib/greet.js', greetJs), + buildFileListing('src/main.js', mainJs), + ]; + expect(expectedFiles[0].hash).not.toBe(getFileHash(actorJson)); + expect(created).toEqual({ + actorId: stored.id, + fullName, + versionNumber: '0.0', + revision: expect.stringMatching(/^[0-9a-f]{16}$/), + files: expectedFiles, + }); + expect(createResult.content?.[1]?.text).toBe( + `Created the private Actor ${fullName}.\nNo build was started; runs use these files only after the version is built.`, + ); + + const read = expectToolSuccess( + await callTool(client, 'get-actor-version', { + actor: created.actorId, + paths: expectedFiles.map(({ path }) => path), + }), + 'get-actor-version', + ); + + expect(read).toEqual({ + actorId: created.actorId, + fullName, + versionNumber: '0.0', + revision: created.revision, + files: created.files, + contents: [ + { path: '.actor/actor.json', content: storedActorJson, encoding: 'utf8' }, + { path: 'assets/logo.png', content: PNG_BASE64, encoding: 'base64' }, + { path: 'assets/raw-bytes', content: NON_UTF8_BYTES.toString('base64'), encoding: 'base64' }, + { path: 'src/lib/greet.js', content: greetJs, encoding: 'utf8' }, + { path: 'src/main.js', content: mainJs, encoding: 'utf8' }, + ], + }); + // A name without the username is resolved against the token's own Actors. + const readByName = expectToolSuccess( + await callTool(client, 'get-actor-version', { actor: name }), + 'get-actor-version', + ); + expect(readByName).toMatchObject({ actorId: created.actorId, fullName, revision: created.revision }); + // autoBuild defaults to false. + expect(await fetchBuildCount(api, created.actorId)).toBe(0); + // build-actor resolves a name without the username the same way. + const buildResult = await callTool(client, 'build-actor', { actor: name, waitSecs: 0 }); + // Taken before the checks, so the build is aborted when one of them fails. + buildId = (buildResult.structuredContent as Partial | undefined)?.build?.id; + const { build } = expectToolSuccess(buildResult, 'build-actor'); + expect(build.actorId).toBe(created.actorId); + } finally { + // A failed abort is only logged, so it cannot hide the error of a failed check, and the Actor is still + // deleted. + if (buildId !== undefined) { + await api + .build(buildId) + .abort() + .catch((error: unknown) => console.error(`Failed to abort build ${buildId}:`, error)); + } + await api.actor(selector).delete(); + } + }), + }, + { + name: 'update-actor-version creates, replaces, edits, and deletes files in one call and leaves the other files, env vars, and build tag as they were', + isDeploymentTest: false, + run: withClient({ tools: ['versions'] }, async (client, ctx) => { + const api = ctx.createApifyClient(); + const name = buildUniqueActorName('update'); + const selector = await fetchOwnActorSelector(api, name); + const readme = '# Source tools test\n'; + const utilJs = 'export const double = (value) => value * 2;\n'; + const mainTs = 'export const greeting = "one";\n'; + const oldJs = 'export const removed = true;\n'; + const zJs = 'z();\n'; + const newJs = 'export const added = true;\n'; + const newReadme = '# Source tools test\n\nReplaced by update-actor-version.\n'; + const editedMainTs = 'export const greeting = "two";\n'; + try { + const seeded = await api.actors().create({ + name, + versions: [ + { + versionNumber: '0.0', + buildTag: 'beta', + sourceType: ActorSourceType.SourceFiles, + envVars: [ + { name: 'MODE', value: 'test' }, + { name: 'SECRET_MODE', value: 'hidden', isSecret: true }, + ], + sourceFiles: [ + { name: 'README.md', format: 'TEXT', content: readme }, + { name: 'assets/logo.png', format: 'BASE64', content: PNG_BASE64 }, + { name: 'src/lib/util.js', format: 'TEXT', content: utilJs }, + // UTF-8 text stored as BASE64, as `apify push` stores a .ts file. + { + name: 'src/main.ts', + format: 'BASE64', + content: Buffer.from(mainTs).toString('base64'), + }, + { name: 'src/old.js', format: 'TEXT', content: oldJs }, + // No format, which the build worker reads as TEXT, and no content, which it reads as + // empty; the casts drop what apify-client's type requires. + { name: 'src/z.js', content: zJs } as ActorVersionSourceFile, + { name: 'src/blank.js', format: 'TEXT' } as ActorVersionSourceFile, + // An empty folder as Console keeps it; the cast adds what apify-client's type leaves out. + { name: 'storage', folder: true } as unknown as ActorVersionSourceFile, + ], + }, + ], + }); + const fullName = `${seeded.username}/${name}`; + // The selector with a slash, which apify-client turns into the API's tilde. + const actorArg = selector.replace('~', '/'); + + const before = expectToolSuccess( + await callTool(client, 'get-actor-version', { actor: actorArg, paths: ['src/main.ts'] }), + 'get-actor-version', + ); + expect(before.files).toEqual([ + buildFileListing('README.md', readme), + buildFileListing('assets/logo.png', Buffer.from(PNG_BASE64, 'base64')), + buildFileListing('src/blank.js', ''), + buildFileListing('src/lib/util.js', utilJs), + buildFileListing('src/main.ts', mainTs), + buildFileListing('src/old.js', oldJs), + buildFileListing('src/z.js', zJs), + ]); + expect(before.contents).toEqual([{ path: 'src/main.ts', content: mainTs, encoding: 'utf8' }]); + const storedBefore = await fetchStoredActor(api, seeded.id); + expect(storedBefore.versions[0].envVars).toEqual([ + expect.objectContaining({ name: 'MODE', value: 'test' }), + expect.objectContaining({ name: 'SECRET_MODE', isSecret: true }), + ]); + + const hashesBefore = new Map(before.files.map(({ path, hash }) => [path, hash])); + const updateArgs = { + actor: actorArg, + expectedRevision: before.revision, + operations: [ + { type: 'write', path: 'src/new.js', content: newJs }, + { + type: 'write', + path: 'README.md', + content: newReadme, + expectedHash: hashesBefore.get('README.md'), + }, + { type: 'edit', path: 'src/main.ts', edits: [{ oldText: '"one"', newText: '"two"' }] }, + { type: 'delete', path: 'src/old.js', expectedHash: hashesBefore.get('src/old.js') }, + ], + }; + const updateResult = await callTool(client, 'update-actor-version', updateArgs); + const updated = expectToolSuccess(updateResult, 'update-actor-version'); + const after = expectToolSuccess( + await callTool(client, 'get-actor-version', { + actor: actorArg, + paths: ['src/main.ts', 'src/new.js'], + }), + 'get-actor-version', + ); + + // The revision update-actor-version reported is the one the next read gets. + expect(updated).toEqual({ + revision: after.revision, + changed: true, + changes: [ + { path: 'README.md', action: 'updated', hash: getFileHash(newReadme) }, + { path: 'src/main.ts', action: 'updated', hash: getFileHash(editedMainTs) }, + { path: 'src/new.js', action: 'created', hash: getFileHash(newJs) }, + { path: 'src/old.js', action: 'deleted' }, + ], + }); + expect(updated.revision).not.toBe(before.revision); + expect(updateResult.content?.[1]?.text).toBe( + `Updated version 0.0 of ${fullName}.\nNo build was started; runs use these files only after the version is built.`, + ); + expect(after.files).toEqual([ + buildFileListing('README.md', newReadme), + buildFileListing('assets/logo.png', Buffer.from(PNG_BASE64, 'base64')), + buildFileListing('src/blank.js', ''), + buildFileListing('src/lib/util.js', utilJs), + buildFileListing('src/main.ts', editedMainTs), + buildFileListing('src/new.js', newJs), + buildFileListing('src/z.js', zJs), + ]); + expect(after.contents).toEqual([ + { path: 'src/main.ts', content: editedMainTs, encoding: 'utf8' }, + { path: 'src/new.js', content: newJs, encoding: 'utf8' }, + ]); + + // Untouched entries, the folder and the ones without format or content included, are stored as they + // were; the edited file stays BASE64. + const storedAfter = await fetchStoredActor(api, seeded.id); + const entriesBefore = new Map( + extractSortedSourceFiles(storedBefore.versions[0]).map((entry) => [entry.name, entry]), + ); + expect(entriesBefore.get('storage')).toMatchObject({ name: 'storage', folder: true }); + expect(entriesBefore.get('src/z.js')).toEqual({ name: 'src/z.js', content: zJs }); + expect(entriesBefore.get('src/blank.js')).toEqual({ name: 'src/blank.js', format: 'TEXT' }); + expect(extractSortedSourceFiles(storedAfter.versions[0])).toEqual([ + expect.objectContaining({ name: 'README.md', format: 'TEXT', content: newReadme }), + entriesBefore.get('assets/logo.png'), + entriesBefore.get('src/blank.js'), + entriesBefore.get('src/lib/util.js'), + expect.objectContaining({ + name: 'src/main.ts', + format: 'BASE64', + content: Buffer.from(editedMainTs).toString('base64'), + }), + expect.objectContaining({ name: 'src/new.js', format: 'TEXT', content: newJs }), + entriesBefore.get('src/z.js'), + entriesBefore.get('storage'), + ]); + // Every other field of the version, env vars and build tag included, is unchanged. + expect({ ...storedAfter.versions[0], sourceFiles: undefined }).toEqual({ + ...storedBefore.versions[0], + sourceFiles: undefined, + }); + expect(storedAfter.versions[0].buildTag).toBe('beta'); + + // A retry of the same call fails on expectedRevision instead of applying the operations twice. + const retryText = expectToolFailure(await callTool(client, 'update-actor-version', updateArgs)); + expect(retryText).toBe( + 'Nothing was written: expectedRevision failed with REVISION_MISMATCH. ' + + `The version's revision is ${updated.revision}, not ${before.revision}.`, + ); + expect(extractWrittenState(await fetchStoredActor(api, seeded.id))).toEqual( + extractWrittenState(storedAfter), + ); + expect(await fetchBuildCount(api, seeded.id)).toBe(0); + } finally { + await api.actor(selector).delete(); + } + }), + }, + { + name: 'update-actor-version fails a write, an edit, or a revision that does not match with its reason code and writes nothing', + isDeploymentTest: false, + run: withClient({ tools: ['versions'] }, async (client, ctx) => { + const api = ctx.createApifyClient(); + const name = buildUniqueActorName('conflicts'); + const selector = await fetchOwnActorSelector(api, name); + const mainJs = 'console.log("one");\n'; + try { + const seeded = await api.actors().create({ + name, + versions: [ + { + versionNumber: '0.0', + sourceType: ActorSourceType.SourceFiles, + sourceFiles: [ + { name: 'README.md', format: 'TEXT', content: '# Conflicts\n' }, + { name: 'src/main.js', format: 'TEXT', content: mainJs }, + ], + }, + ], + }); + const listing = expectToolSuccess( + await callTool(client, 'get-actor-version', { actor: seeded.id }), + 'get-actor-version', + ); + const mainHash = getFileHash(mainJs); + expect(listing.files.find(({ path }) => path === 'src/main.js')?.hash).toBe(mainHash); + const storedBefore = extractWrittenState(await fetchStoredActor(api, seeded.id)); + // Each failing operation follows a write that succeeds alone, which must not be saved either. + const addFile = { type: 'write', path: 'src/added.js', content: 'export {};\n' }; + const conflicts = [ + { + operations: [addFile, { type: 'write', path: 'src/main.js', content: 'console.log("x");\n' }], + text: + 'Nothing was written: operations[1] (write src/main.js) failed with FILE_EXISTS. ' + + `src/main.js exists with hash ${mainHash}; a write with that hash as expectedHash replaces it.`, + }, + { + operations: [ + addFile, + { + type: 'write', + path: 'src/main.js', + content: 'console.log("x");\n', + expectedHash: WRONG_HASH, + }, + ], + text: + 'Nothing was written: operations[1] (write src/main.js) failed with HASH_MISMATCH. ' + + `src/main.js has hash ${mainHash}, not ${WRONG_HASH}.`, + }, + { + // The second edit runs on the text the first left, which no longer has "one". + operations: [ + addFile, + { + type: 'edit', + path: 'src/main.js', + edits: [ + { oldText: '"one"', newText: '"two"' }, + { oldText: '"one"', newText: '"three"' }, + ], + }, + ], + text: + 'Nothing was written: operations[1] (edit src/main.js) failed with NO_MATCH. ' + + 'oldText of edits[1] is not in the file.', + }, + { + expectedRevision: WRONG_HASH, + operations: [addFile], + text: + 'Nothing was written: expectedRevision failed with REVISION_MISMATCH. ' + + `The version's revision is ${listing.revision}, not ${WRONG_HASH}.`, + }, + ]; + + for (const { text, ...args } of conflicts) { + const result = await callTool(client, 'update-actor-version', { actor: seeded.id, ...args }); + + expect(expectToolFailure(result)).toBe(text); + expect(extractWrittenState(await fetchStoredActor(api, seeded.id))).toEqual(storedBefore); + } + const reread = expectToolSuccess( + await callTool(client, 'get-actor-version', { actor: seeded.id }), + 'get-actor-version', + ); + expect(reread.revision).toBe(listing.revision); + } finally { + await api.actor(selector).delete(); + } + }), + }, + { + name: 'update-actor-version refuses a version stored in a Git repository, names the repository without its credentials, and changes nothing', + isDeploymentTest: false, + run: withClient({ tools: ['versions'] }, async (client, ctx) => { + const api = ctx.createApifyClient(); + const name = buildUniqueActorName('git'); + const selector = await fetchOwnActorSelector(api, name); + // A placeholder user and password, which the API returns to the owner and the refusal must not repeat. + const gitRepoUrl = 'https://ci:not-a-secret@github.com/apify/actor-templates.git#master'; + try { + const seeded = await api.actors().create({ + name, + versions: [{ versionNumber: '0.0', sourceType: ActorSourceType.GitRepo, gitRepoUrl }], + }); + const storedBefore = await fetchStoredActor(api, seeded.id); + expect(storedBefore.versions).toEqual([ + expect.objectContaining({ versionNumber: '0.0', sourceType: 'GIT_REPO', gitRepoUrl }), + ]); + const refusal = + `Version 0.0 of ${seeded.username}/${name} has its files in the Git repository ` + + 'https://github.com/apify/actor-templates.git#master, not stored on Apify, so this tool cannot ' + + 'work on them; use the repository.'; + + const updateResult = await callTool(client, 'update-actor-version', { + actor: seeded.id, + operations: [{ type: 'write', path: 'src/main.js', content: 'console.log("one");\n' }], + }); + const readResult = await callTool(client, 'get-actor-version', { actor: seeded.id }); + + expect(expectToolFailure(updateResult)).toBe(refusal); + expect(expectToolFailure(readResult)).toBe(refusal); + expect(extractWrittenState(await fetchStoredActor(api, seeded.id))).toEqual( + extractWrittenState(storedBefore), + ); + } finally { + await api.actor(selector).delete(); + } + }), + }, + { + name: 'create-actor returns the platform error for a name the account already uses and creates nothing', + isDeploymentTest: false, + run: withClient({ tools: ['versions'] }, async (client, ctx) => { + const api = ctx.createApifyClient(); + const name = buildUniqueActorName('taken'); + const selector = await fetchOwnActorSelector(api, name); + try { + const seeded = await api.actors().create({ + name, + versions: [ + { + versionNumber: '0.0', + sourceType: ActorSourceType.SourceFiles, + sourceFiles: [{ name: 'src/main.js', format: 'TEXT', content: 'console.log("first");\n' }], + }, + ], + }); + const storedBefore = extractWrittenState(await fetchStoredActor(api, seeded.id)); + + const result = await callTool(client, 'create-actor', { + name, + files: [{ path: 'src/main.js', content: 'console.log("second");\n' }], + }); + + // The sentence is the platform's, so only the name, the API error type, and the tool's frame are + // exact. In apify-core it reads `Some other Actor already has this name ("").`, with status 409. + expect(expectToolFailure(result)).toMatch( + new RegExp( + `^Error calling tool "create-actor": .*"${name}".* \\(API error type: actor-name-not-unique\\)\\.$`, + ), + ); + // The name still leads to the first Actor, stored as it was. + const storedAfter = await fetchStoredActor(api, selector); + expect(storedAfter.id).toBe(seeded.id); + expect(extractWrittenState(storedAfter)).toEqual(storedBefore); + } finally { + await api.actor(selector).delete(); + } + }), + }, +]; diff --git a/tests/test_kit/helpers.ts b/tests/test_kit/helpers.ts index 996b98c8b..b66485181 100644 --- a/tests/test_kit/helpers.ts +++ b/tests/test_kit/helpers.ts @@ -27,7 +27,7 @@ export const ACTOR_EXAMPLE_MCP_SERVER = 'apify/example-mcp-server'; const STDIO_TRANSPORT: Transport = 'stdio'; -export const RETIRED_SELECTORS = ['add-actor', 'experimental', 'preview', 'get-actor-log'] as const; +export const RETIRED_SELECTORS = ['add-actor', 'experimental', 'preview', 'get-actor-log', 'source'] as const; export const AUTO_INJECTED_TOOL_NAMES = AUTO_INJECTED_TOOLS.map((t) => t.name); export const DEFAULT_ACTOR_NAMES = defaults.actors.map((actor) => actorNameToToolName(actor)); diff --git a/tests/test_kit/index.ts b/tests/test_kit/index.ts index 7242995f6..35c12576d 100644 --- a/tests/test_kit/index.ts +++ b/tests/test_kit/index.ts @@ -23,6 +23,7 @@ export { schedulesCases } from './cases/schedules.cases.js'; export { storageCases } from './cases/storage.cases.js'; export { tasksCases } from './cases/tasks.cases.js'; export { toolsCases } from './cases/tools.cases.js'; +export { versionsCases } from './cases/versions.cases.js'; import { actorsCases } from './cases/actors.cases.js'; import { appsCases } from './cases/apps.cases.js'; @@ -32,6 +33,7 @@ import { schedulesCases } from './cases/schedules.cases.js'; import { storageCases } from './cases/storage.cases.js'; import { tasksCases } from './cases/tasks.cases.js'; import { toolsCases } from './cases/tools.cases.js'; +import { versionsCases } from './cases/versions.cases.js'; import type { Case } from './types.js'; /** All cases from every capability group. */ @@ -43,5 +45,6 @@ export const allCases: Case[] = [ ...tasksCases, ...storageCases, ...schedulesCases, + ...versionsCases, ...paymentsCases, ]; diff --git a/tests/unit/tools.build_actor.test.ts b/tests/unit/tools.build_actor.test.ts index 0c39cf744..749f630ac 100644 --- a/tests/unit/tools.build_actor.test.ts +++ b/tests/unit/tools.build_actor.test.ts @@ -155,14 +155,68 @@ describe('build-actor', () => { expectSchemaConformingStructuredContent(result, buildActorToolOutputSchema); }); - it('returns a not-found error when the Actor does not exist', async () => { - getMock.mockResolvedValue(undefined); + describe('Actor lookup', () => { + it('looks up a bare name once, as the name of one of your own Actors', async () => { + await callTool({ actor: 'my-actor' }); - const { text, structuredContent } = await callToolExpectingUserError({ actor: 'john/missing' }); + // The second call starts the build by the resolved Actor ID. + expect(actorMock.mock.calls).toEqual([['~my-actor'], ['actor-1']]); + expect(buildMock).toHaveBeenCalledWith('0.1', { useCache: true }); + }); - expect(text).toBe("Actor 'john/missing' not found."); - expect(structuredContent).toBeUndefined(); - expect(buildMock).not.toHaveBeenCalled(); + it('looks up a bare value shaped like an ID as a name when no Actor has that ID', async () => { + getMock.mockResolvedValueOnce(undefined).mockResolvedValueOnce(mockActor()); + + await callTool({ actor: 'myscraperversion2' }); + + expect(actorMock.mock.calls).toEqual([['myscraperversion2'], ['~myscraperversion2'], ['actor-1']]); + expect(buildMock).toHaveBeenCalledTimes(1); + }); + + it.each(['qGXMy0NAkWsIIb9LZ', 'john/my-actor', 'john~my-actor'])( + 'looks up %s once, as given', + async (actor) => { + await callTool({ actor }); + + expect(actorMock.mock.calls).toEqual([[actor], ['actor-1']]); + }, + ); + + it.each([ + ['a bare name', 'missing', ['~missing']], + ['a bare value shaped like an ID', 'myscraperversion2', ['myscraperversion2', '~myscraperversion2']], + ['a full name', 'john/missing', ['john/missing']], + ['a full name with a tilde', 'john~missing', ['john~missing']], + ])('returns a not-found error for %s that matches no Actor', async (_, actor, lookups) => { + getMock.mockResolvedValue(undefined); + + const { text, structuredContent } = await callToolExpectingUserError({ actor }); + + expect(actorMock.mock.calls).toEqual(lookups.map((lookup) => [lookup])); + expect(text).toBe( + `Actor '${actor}' not found. The actor argument takes an ID, a name (resolved against your own Actors), or a full name as username/name.`, + ); + expect(structuredContent).toBeUndefined(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it.each([ + ['a run', 'john/my-actor/runs/last', { id: 'run-1', actId: 'actor-1', status: 'SUCCEEDED' }], + // Has a name but no username. + ['an env var', 'john/my-actor/versions/0.1/env-vars/API_KEY', { name: 'API_KEY', value: 'secret-value' }], + // Has a username but no name, as a run's default storage does. + ["a run's key-value store", 'john/my-actor/runs/last/key-value-store', { id: 'kvs-1', username: 'john' }], + ])('reports %s, reached by extra path segments, as a missing Actor', async (_, actor, document) => { + getMock.mockResolvedValue(document); + + const { text, structuredContent } = await callToolExpectingUserError({ actor }); + + expect(text).toBe( + `Actor '${actor}' not found. The actor argument takes an ID, a name (resolved against your own Actors), or a full name as username/name.`, + ); + expect(structuredContent).toBeUndefined(); + expect(buildMock).not.toHaveBeenCalled(); + }); }); it('returns an error when the Actor has no versions', async () => { diff --git a/tests/unit/tools.categories.test.ts b/tests/unit/tools.categories.test.ts index 43edbe91e..f895fa0c1 100644 --- a/tests/unit/tools.categories.test.ts +++ b/tests/unit/tools.categories.test.ts @@ -72,7 +72,7 @@ describe('getCategoryTools', () => { 'tasks', 'schedules', 'builds', - 'source', + 'versions', 'api', 'dev', ]; diff --git a/tests/unit/tools.create_actor.test.ts b/tests/unit/tools.create_actor.test.ts new file mode 100644 index 000000000..5606ec2e1 --- /dev/null +++ b/tests/unit/tools.create_actor.test.ts @@ -0,0 +1,906 @@ +import { createHash } from 'node:crypto'; + +import { ApifyApiError } from 'apify-client'; +import type { AxiosResponse } from 'axios'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; + +import { HELPER_TOOLS } from '../../src/const.js'; +import { prepareToolCallContext } from '../../src/payments/helpers.js'; +import { createActorToolOutputSchema } from '../../src/tools/structured_output_schemas.js'; +import { createActor } from '../../src/tools/versions/create_actor.js'; +import { getActorVersion } from '../../src/tools/versions/get_actor_version.js'; +import { buildFilesRevision } from '../../src/tools/versions/source_files.js'; +import type { HelperTool, InternalToolArgs } from '../../src/types.js'; +import { VERBATIM_LINKS_NUDGE } from '../../src/utils/console_link.js'; +import { getUserInfoCached } from '../../src/utils/userid_cache.js'; +import { + expectSchemaConformingStructuredContent, + expectSoftFailInvalidInput, + mockUserInfo, + stubToolCallContext, + type TextToolResult, + type ToolTelemetrySnapshot, +} from './helpers/tool_context.js'; + +vi.mock('../../src/utils/userid_cache.js', () => ({ + getUserInfoCached: vi.fn(), +})); + +const actorsCreateMock = vi.fn(); +const buildMock = vi.fn(); +const actorGetMock = vi.fn(); +const actorMock = vi.fn(() => ({ build: buildMock, get: actorGetMock })); +// The build client, which only waiting for a build uses. +const buildClientMock = vi.fn(); + +const stubClient = { + actors: () => ({ create: actorsCreateMock }), + actor: actorMock, + build: buildClientMock, +} as unknown as InternalToolArgs['apifyClient']; + +/** Already names the Actor, so the platform stores it unchanged. */ +const ACTOR_JSON = { path: '.actor/actor.json', content: '{"actorSpecification": 1, "name": "my-actor"}' }; +const DOCKERFILE = { path: 'Dockerfile', content: 'FROM apify/actor-node:20\n' }; +const MAIN_JS = { path: 'src/main.js', content: 'console.log(1);\n' }; + +type CreateOutput = { + actorId: string; + fullName: string; + versionNumber: string; + revision: string; + files: { path: string; sizeBytes: number; hash: string }[]; + warnings?: string[]; + build?: Record; + buildError?: string; +}; + +type CreateResult = TextToolResult & { structuredContent: CreateOutput; toolTelemetry?: ToolTelemetrySnapshot }; + +type SentVersion = { + versionNumber?: string; + buildTag?: string; + sourceFiles: { name: string; format: string; content: string }[]; +}; + +/** + * What the platform stores on create: the only version gets the build tag latest when none is sent, and + * `.actor/actor.json` gets `name` set to the Actor name, or stays as sent when it is not JSON. The version number is + * not defaulted: the platform refuses a version without one. + */ +function buildStoredVersions(body: { name: string; versions: SentVersion[] }) { + return body.versions.map((version) => ({ + buildTag: 'latest', + ...version, + sourceFiles: version.sourceFiles.map((file) => { + if (file.name !== '.actor/actor.json') return file; + let config: { name?: string }; + try { + config = JSON.parse(file.content) as { name?: string }; + } catch { + return file; + } + if (config.name === body.name) return file; + return { ...file, content: JSON.stringify({ ...config, name: body.name }, null, 4) }; + }), + })); +} + +function sha256Prefix(data: Buffer | string): string { + return createHash('sha256').update(data).digest('hex').slice(0, 16); +} + +/** `attempt` counts apify-client's tries of the request, retries included. */ +function apiError(status: number, message: string, type = 'some-error', attempt = 1): ApifyApiError { + return new ApifyApiError({ data: { error: { type, message } }, status } as AxiosResponse, attempt); +} + +async function callTool(args: Record, signal?: AbortSignal): Promise { + const context = stubToolCallContext({ name: 'my-actor', ...args }, stubClient); + const withSignal = signal === undefined ? context : { ...context, signal }; + return (await (createActor as HelperTool).call(withSignal)) as CreateResult; +} + +/** The listing get-actor-version returns for the version the platform stored from the POST. */ +async function readCreatedVersion(): Promise<{ revision: string; files: unknown[] }> { + const [body] = actorsCreateMock.mock.calls[0] as [{ name: string; versions: SentVersion[] }]; + actorGetMock.mockResolvedValue({ + id: 'actor-9', + name: 'my-actor', + username: 'john', + versions: buildStoredVersions(body), + }); + const read = (await (getActorVersion as HelperTool).call( + stubToolCallContext({ actor: 'john/my-actor' }, stubClient), + )) as { structuredContent: { revision: string; files: unknown[] } }; + return read.structuredContent; +} + +function getSentVersion(): SentVersion & Record { + const [body] = actorsCreateMock.mock.calls[0] as [{ versions: (SentVersion & Record)[] }]; + return body.versions[0]; +} + +describe('create-actor', () => { + beforeEach(() => { + vi.clearAllMocks(); + actorsCreateMock.mockImplementation(async (body: { name: string; versions: SentVersion[] }) => ({ + id: 'actor-9', + name: body.name, + username: 'john', + versions: buildStoredVersions(body), + })); + buildMock.mockResolvedValue({ + id: 'build-1', + actId: 'actor-9', + buildNumber: '0.0.1', + status: 'READY', + startedAt: new Date('2026-09-01T10:00:00.000Z'), + }); + }); + + describe('input schema', () => { + const validate = (args: Record) => (createActor as HelperTool).ajvValidate(args); + const file = { path: 'src/main.js', content: 'x' }; + + it('requires only name and files', () => { + expect((createActor as HelperTool).inputSchema.required).toEqual(['name', 'files']); + }); + + it('keeps every field it declares and strips unknown keys', () => { + const args = { + name: 'my-actor', + title: 'My Actor', + description: '', + files: [{ ...file, encoding: 'base64', extra: 1 }], + versionNumber: '1.0', + buildTag: 'beta', + autoBuild: true, + unknown: 'x', + }; + + expect(validate(args)).toBe(true); + expect(args).toStrictEqual({ + name: 'my-actor', + title: 'My Actor', + description: '', + files: [{ ...file, encoding: 'base64' }], + versionNumber: '1.0', + buildTag: 'beta', + autoBuild: true, + }); + }); + + it.each<[string, Record]>([ + ['no name', { files: [file] }], + ['an empty name', { name: '', files: [file] }], + ['no files', { name: 'x' }], + ['empty files', { name: 'x', files: [] }], + ['an empty title', { name: 'x', title: '', files: [file] }], + ['an empty buildTag', { name: 'x', buildTag: '', files: [file] }], + ['an empty path', { name: 'x', files: [{ path: '', content: 'x' }] }], + ['a file without a path', { name: 'x', files: [{ content: 'x' }] }], + ['a file without content', { name: 'x', files: [{ path: 'a.js' }] }], + ['an unknown encoding', { name: 'x', files: [{ ...file, encoding: 'hex' }] }], + ])('rejects %s', (_, args) => { + expect(validate(args)).toBe(false); + }); + + it('accepts one empty file', () => { + expect(validate({ name: 'x', files: [{ path: 'a.js', content: '' }] })).toBe(true); + }); + }); + + describe('logged arguments', () => { + const SECRET = 'synthetic-secret-4f2a'; + const getLoggedArgs = (args: Record) => + prepareToolCallContext({ provider: undefined, tool: createActor, args, apifyToken: 'test-token' }) + .toolArgsRedacted; + + it('keep the paths and other metadata, and replace file content with a marker', () => { + const args = { + name: 'my-actor', + title: 'My Actor', + description: 'Scrapes pages', + versionNumber: '1.0', + buildTag: 'beta', + autoBuild: true, + files: [ + { path: 'src/config.js', content: `export const key = '${SECRET}';\n`, encoding: 'utf8' }, + { path: 'assets/key.bin', content: Buffer.from(SECRET).toString('base64'), extra: SECRET }, + ], + undeclared: SECRET, + }; + const argsBefore = structuredClone(args); + + const logged = getLoggedArgs(args); + + expect(JSON.stringify(logged)).not.toContain(SECRET); + expect(JSON.stringify(logged)).not.toContain(Buffer.from(SECRET).toString('base64')); + expect(logged).toStrictEqual({ + name: 'my-actor', + title: 'My Actor', + description: 'Scrapes pages', + versionNumber: '1.0', + buildTag: 'beta', + autoBuild: true, + files: [ + { path: 'src/config.js', encoding: 'utf8', content: '[REDACTED]' }, + { path: 'assets/key.bin', content: '[REDACTED]' }, + ], + }); + expect(args).toStrictEqual(argsBefore); + }); + + it.each<[string, Record]>([ + ['no arguments', {}], + ['files that are not a list', { files: SECRET }], + ['files that are not objects', { files: [SECRET, null, 1, [SECRET]] }], + ['a path that is not a string', { files: [{ path: { text: SECRET }, content: 'x' }] }], + ['content that is not a string', { files: [{ path: 'a.js', content: { text: SECRET } }] }], + ['a name that is not a string', { name: [SECRET], files: [] }], + ])('accept %s without throwing or logging their values', (_, args) => { + const argsBefore = structuredClone(args); + + expect(JSON.stringify(getLoggedArgs(args))).not.toContain(SECRET); + expect(args).toStrictEqual(argsBefore); + }); + }); + + it('is a non-destructive, non-idempotent, open-world tool without payment, since autoBuild starts a build', () => { + expect(createActor.name).toBe(HELPER_TOOLS.ACTOR_CREATE); + expect(createActor.annotations).toEqual({ + title: 'Create Actor', + readOnlyHint: false, + destructiveHint: false, + idempotentHint: false, + openWorldHint: true, + }); + expect((createActor as HelperTool).paymentRequired).toBeUndefined(); + }); + + it('creates the Actor and version 0.0 in one POST, with no env vars and no build tag', async () => { + const result = await callTool({ + title: 'My Actor', + description: 'Does things.', + files: [MAIN_JS, ACTOR_JSON, DOCKERFILE], + }); + + expectSchemaConformingStructuredContent(result, createActorToolOutputSchema); + expect(actorsCreateMock).toHaveBeenCalledTimes(1); + expect(actorsCreateMock).toHaveBeenCalledWith({ + name: 'my-actor', + title: 'My Actor', + description: 'Does things.', + versions: [ + { + versionNumber: '0.0', + sourceType: 'SOURCE_FILES', + sourceFiles: [ + { name: 'src/main.js', format: 'TEXT', content: MAIN_JS.content }, + { name: '.actor/actor.json', format: 'TEXT', content: ACTOR_JSON.content }, + { name: 'Dockerfile', format: 'TEXT', content: DOCKERFILE.content }, + ], + }, + ], + }); + const files = [ACTOR_JSON, DOCKERFILE, MAIN_JS].map(({ path, content }) => ({ + path, + sizeBytes: content.length, + hash: sha256Prefix(content), + })); + expect(result.structuredContent).toEqual({ + actorId: 'actor-9', + fullName: 'john/my-actor', + versionNumber: '0.0', + revision: buildFilesRevision(files), + files, + }); + expect(result.content).toHaveLength(2); + expect(JSON.parse(result.content[0].text)).toEqual(result.structuredContent); + expect(result.content[1].text).toBe( + 'Created the private Actor john/my-actor.\nNo build was started; runs use these files only after the version is built.', + ); + // No other call: nothing reads or updates an existing Actor, and no build starts. + expect(actorMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('sends only the name and the version when title and description are not given', async () => { + await callTool({ files: [MAIN_JS] }); + + expect(actorsCreateMock.mock.calls).toStrictEqual([ + [ + { + name: 'my-actor', + versions: [ + { + versionNumber: '0.0', + sourceType: 'SOURCE_FILES', + sourceFiles: [{ name: 'src/main.js', format: 'TEXT', content: MAIN_JS.content }], + }, + ], + }, + ], + ]); + }); + + it('sends an empty description as given', async () => { + await callTool({ description: '', files: [MAIN_JS] }); + + expect(actorsCreateMock.mock.calls).toStrictEqual([ + [ + { + name: 'my-actor', + description: '', + versions: [ + { + versionNumber: '0.0', + sourceType: 'SOURCE_FILES', + sourceFiles: [{ name: 'src/main.js', format: 'TEXT', content: MAIN_JS.content }], + }, + ], + }, + ], + ]); + }); + + it('sends versionNumber and buildTag when given', async () => { + const result = await callTool({ versionNumber: '1.2', buildTag: 'beta', files: [ACTOR_JSON] }); + + expect(actorsCreateMock.mock.calls).toStrictEqual([ + [ + { + name: 'my-actor', + versions: [ + { + versionNumber: '1.2', + buildTag: 'beta', + sourceType: 'SOURCE_FILES', + sourceFiles: [{ name: '.actor/actor.json', format: 'TEXT', content: ACTOR_JSON.content }], + }, + ], + }, + ], + ]); + expect(result.structuredContent.versionNumber).toBe('1.2'); + }); + + it('returns the hashes and revision of the files as stored, which a later read returns', async () => { + const unnamedConfig = { path: '.actor/actor.json', content: '{"actorSpecification": 1}' }; + const result = await callTool({ files: [unnamedConfig, MAIN_JS] }); + const storedConfig = JSON.stringify({ actorSpecification: 1, name: 'my-actor' }, null, 4); + const [body] = actorsCreateMock.mock.calls[0] as [{ name: string; versions: SentVersion[] }]; + actorGetMock.mockResolvedValue({ + id: 'actor-9', + name: 'my-actor', + username: 'john', + versions: buildStoredVersions(body), + }); + const read = (await (getActorVersion as HelperTool).call( + stubToolCallContext({ actor: 'john/my-actor' }, stubClient), + )) as { structuredContent: { revision: string; files: unknown[] } }; + + expect(result.structuredContent.files[0]).toEqual({ + path: '.actor/actor.json', + sizeBytes: storedConfig.length, + hash: sha256Prefix(storedConfig), + }); + expect(result.structuredContent.revision).toBe(read.structuredContent.revision); + expect(result.structuredContent.files).toEqual(read.structuredContent.files); + }); + + it('stores binary files as BASE64 and lists them by their decoded bytes', async () => { + const bytes = Buffer.from([137, 80, 78, 71]); + const readme = Buffer.from('hello\n').toString('base64'); + const result = await callTool({ + files: [ + { path: 'assets/logo.png', content: bytes.toString('base64') }, + { path: 'README.md', content: readme, encoding: 'base64' }, + ], + }); + + expect(getSentVersion().sourceFiles).toEqual([ + { name: 'assets/logo.png', format: 'BASE64', content: bytes.toString('base64') }, + { name: 'README.md', format: 'BASE64', content: readme }, + ]); + expect(result.structuredContent.files).toContainEqual({ + path: 'assets/logo.png', + sizeBytes: 4, + hash: sha256Prefix(bytes), + }); + }); + + it.each([ + ['assets/icon.png', undefined, 'BASE64'], + ['assets/ICON.PNG', undefined, 'BASE64'], + ['fonts/a.woff2', undefined, 'BASE64'], + ['src/main.ts', undefined, 'TEXT'], + ['assets/icon.svg', undefined, 'TEXT'], + ['Dockerfile', undefined, 'TEXT'], + ['assets/icon.png', 'utf8', 'TEXT'], + ['README.md', 'base64', 'BASE64'], + ])('stores %s with encoding %s as %s, with the content as sent', async (path, encoding, format) => { + const content = 'aGk='; + + await callTool({ files: [{ path, content, ...(encoding !== undefined && { encoding }) }] }); + + expect(getSentVersion().sourceFiles).toStrictEqual([{ name: path, format, content }]); + }); + + it('sends utf8 content byte for byte', async () => { + const content = '\uFEFFline one \r\nnázev = "🙂"\n\tend\r'; + + await callTool({ files: [{ path: 'src/main.js', content, encoding: 'utf8' }] }); + + expect(getSentVersion().sourceFiles).toStrictEqual([{ name: 'src/main.js', format: 'TEXT', content }]); + }); + + it('sends non-ASCII paths as given, NFC and NFD forms as two files, with the hashes and revision a later read returns', async () => { + const nfcPath = 'src/n\u00e1zev.js'; + const nfdPath = 'src/na\u0301zev.js'; + + const result = await callTool({ + files: [ + { path: nfcPath, content: 'x' }, + { path: nfdPath, content: 'y' }, + ], + }); + + expect(getSentVersion().sourceFiles).toStrictEqual([ + { name: nfcPath, format: 'TEXT', content: 'x' }, + { name: nfdPath, format: 'TEXT', content: 'y' }, + ]); + const read = await readCreatedVersion(); + expect(result.structuredContent.files).toEqual(read.files); + expect(result.structuredContent.revision).toBe(read.revision); + // NFD sorts first: its 'a' comes before the NFC 'á'. + expect(result.structuredContent.files.map(({ path }) => path)).toEqual([nfdPath, nfcPath]); + }); + + it('normalizes paths the way get-actor-version lists them', async () => { + const result = await callTool({ + files: [ + { path: './src//main.js', content: 'x' }, + { path: 'src/../README.md', content: 'y' }, + { path: 'src/./lib/util.js', content: 'z' }, + ], + }); + + expect(getSentVersion().sourceFiles).toEqual([ + { name: 'src/main.js', format: 'TEXT', content: 'x' }, + { name: 'README.md', format: 'TEXT', content: 'y' }, + { name: 'src/lib/util.js', format: 'TEXT', content: 'z' }, + ]); + expect(result.structuredContent.files.map(({ path }) => path)).toEqual([ + 'README.md', + 'src/lib/util.js', + 'src/main.js', + ]); + }); + + it.each(['/src/main.js', '..', '../x.js', 'src/../../x.js'])( + 'refuses a file at %s, outside the Actor root, which the build refuses, and creates nothing', + async (path) => { + const result = await callTool({ autoBuild: true, files: [MAIN_JS, { path, content: 'x' }] }); + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe( + `files[1] (${path}) has a path outside the Actor root; paths are relative to it, such as src/main.js.`, + ); + expect(actorsCreateMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }, + ); + + it.each(['src\\main.js', '.\\Dockerfile'])( + 'refuses a file at %s, whose backslash the build takes as part of the file name, and creates nothing', + async (path) => { + const result = await callTool({ autoBuild: true, files: [MAIN_JS, { path, content: 'x' }] }); + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe( + `files[1] (${path}) has a backslash in its path; paths use / as the separator.`, + ); + expect(actorsCreateMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }, + ); + + it.each([ + ['U+0000', 'src/a\0.js'], + ['U+000A', 'src/\nmain.js'], + ['U+0009', 'src/a\tb.js'], + ['U+001F', 'src/a\u001f.js'], + ['U+007F', 'src/a\u007f.js'], + ])('refuses a path with the control character %s, and creates nothing', async (character, path) => { + const result = await callTool({ autoBuild: true, files: [MAIN_JS, { path, content: 'x' }] }); + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe( + `files[1] has the control character ${character} in its path; ` + + 'a path cannot hold characters U+0000 to U+001F or U+007F.', + ); + expect(actorsCreateMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('refuses a path with a lone UTF-16 surrogate, and creates nothing', async () => { + const result = await callTool({ autoBuild: true, files: [MAIN_JS, { path: 'src/a\uD83D.js', content: 'x' }] }); + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe( + 'files[1] has a lone UTF-16 surrogate in its path, which UTF-8 cannot store.', + ); + expect(actorsCreateMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it.each(['.', 'src/', 'src/..'])( + 'refuses a file at %s, which names a folder, and creates nothing', + async (path) => { + const result = await callTool({ files: [{ path, content: 'x' }] }); + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe(`files[0] (${path}) has a path that names a folder, not a file.`); + expect(actorsCreateMock).not.toHaveBeenCalled(); + }, + ); + + it.each([ + [ + 'a file under another file', + [{ path: 'src', content: 'x' }, MAIN_JS], + 'files[1] (src/main.js) collides with src', + ], + [ + 'a file at the folder of another file', + [MAIN_JS, { path: 'src', content: 'x' }], + 'files[1] (src) collides with src/main.js', + ], + [ + 'a file two folders under another file', + [ + { path: 'src', content: 'x' }, + { path: 'src/lib/util.js', content: 'x' }, + ], + 'files[1] (src/lib/util.js) collides with src', + ], + [ + 'a file at a folder two levels above another file', + [ + { path: 'src/lib/util.js', content: 'x' }, + { path: 'src', content: 'x' }, + ], + 'files[1] (src) collides with src/lib/util.js', + ], + [ + 'a file at a nested folder of an earlier file', + [MAIN_JS, { path: 'src/lib/util.js', content: 'x' }, { path: 'src/lib', content: 'x' }], + 'files[2] (src/lib) collides with src/lib/util.js', + ], + ])('refuses %s, since a path cannot be both a file and a folder, and creates nothing', async (_, files, prefix) => { + const result = await callTool({ files }); + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe(`${prefix}; one path cannot be both a file and a folder.`); + expect(actorsCreateMock).not.toHaveBeenCalled(); + }); + + it('checks 20,000 files for repeated and colliding paths within a second', async () => { + const files = Array.from({ length: 20_000 }, (_, i) => ({ path: `src/d${i % 50}/f${i}.js`, content: 'x' })); + + const startedAt = performance.now(); + await callTool({ files }); + + expect(performance.now() - startedAt).toBeLessThan(1000); + expect(getSentVersion().sourceFiles).toHaveLength(20_000); + }); + + it.each([ + ['a.js', './a.js', 'a.js'], + ['.actor/actor.json', '.actor//actor.json', '.actor/actor.json'], + ['src/main.js', 'src/main.js', 'src/main.js'], + ])('refuses %s and %s, two files at the same path, and creates nothing', async (first, second, path) => { + const result = await callTool({ + files: [ + { path: first, content: '1' }, + { path: second, content: '2' }, + ], + }); + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe( + `files[1] (${second}) repeats the path ${path}; each path can be sent only once.`, + ); + expect(actorsCreateMock).not.toHaveBeenCalled(); + }); + + it.each([ + ['base64 that is a data URI', { path: 'assets/a.png', content: 'data:image/png;base64,iVBORw0KGgo=' }], + ['text sent to a binary extension', { path: 'assets/a.png', content: 'hello world' }], + ['text sent with encoding base64', { path: 'README.md', content: '# Title\n', encoding: 'base64' }], + ])('refuses %s, which would be stored corrupted, and creates nothing', async (_, file) => { + const result = await callTool({ files: [MAIN_JS, file] }); + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe( + `files[1] (${file.path}) has content that is not valid base64; encoding base64 takes binary content as base64, and text takes encoding utf8.`, + ); + expect(actorsCreateMock).not.toHaveBeenCalled(); + }); + + it('refuses text with a lone UTF-16 surrogate, and creates nothing', async () => { + const result = await callTool({ files: [{ path: 'src/a.js', content: 'smile \uD83D' }] }); + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe( + 'files[0] (src/a.js) has text with a lone UTF-16 surrogate, which UTF-8 cannot store.', + ); + expect(actorsCreateMock).not.toHaveBeenCalled(); + }); + + it('warns about empty files, which the build skips', async () => { + const result = await callTool({ files: [ACTOR_JSON, { path: 'src/__init__.py', content: '' }] }); + + expectSchemaConformingStructuredContent(result, createActorToolOutputSchema); + expect(result.structuredContent.warnings).toEqual([ + 'These files are empty, and the build skips empty files, so they will not exist in the build: src/__init__.py.', + ]); + }); + + it('creates nothing when the request is cancelled before the POST', async () => { + const controller = new AbortController(); + controller.abort(); + + const result = await callTool({ files: [ACTOR_JSON] }, controller.signal); + + expect(actorsCreateMock).not.toHaveBeenCalled(); + expect(result).toEqual({}); + }); + + it('keeps the Actor and starts no build when the request is cancelled during the POST', async () => { + const controller = new AbortController(); + const create = actorsCreateMock.getMockImplementation(); + actorsCreateMock.mockImplementation(async (body: { name: string; versions: SentVersion[] }) => { + controller.abort(); + return create?.(body); + }); + + const result = await callTool({ autoBuild: true, files: [ACTOR_JSON] }, controller.signal); + + expect(result).toEqual({}); + expect(actorsCreateMock).toHaveBeenCalledTimes(1); + expect(actorMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('keeps the Actor and starts no build when the request is cancelled during the Console link lookup', async () => { + const controller = new AbortController(); + vi.mocked(getUserInfoCached).mockImplementationOnce(async () => { + controller.abort(); + return mockUserInfo(); + }); + + const result = await (createActor as HelperTool).call({ + ...stubToolCallContext({ name: 'my-actor', autoBuild: true, files: [ACTOR_JSON] }, stubClient), + apifyToken: 'apify_ui_test', + signal: controller.signal, + }); + + expect(result).toEqual({}); + expect(getUserInfoCached).toHaveBeenCalledTimes(1); + expect(actorsCreateMock).toHaveBeenCalledTimes(1); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('keeps the build it started when the request is cancelled during the build start', async () => { + const controller = new AbortController(); + buildMock.mockImplementationOnce(async () => { + controller.abort(); + return { id: 'build-1', actId: 'actor-9', buildNumber: '0.0.1', status: 'RUNNING' }; + }); + // The build client is what aborts a build. + const abortMock = vi.fn(async () => ({})); + buildClientMock.mockReturnValueOnce({ abort: abortMock }); + + const result = await callTool({ autoBuild: true, files: [ACTOR_JSON] }, controller.signal); + + expect(buildMock).toHaveBeenCalledTimes(1); + expect(buildClientMock).not.toHaveBeenCalled(); + expect(abortMock).not.toHaveBeenCalled(); + expect(result.structuredContent.build).toEqual(expect.objectContaining({ id: 'build-1', status: 'RUNNING' })); + }); + + it('lets an API error through unchanged, such as a name already taken, with no other call', async () => { + const error = apiError(409, 'Some other Actor already has this name.', 'actor-name-not-unique'); + actorsCreateMock.mockRejectedValue(error); + + await expect(callTool({ autoBuild: true, files: [ACTOR_JSON] })).rejects.toBe(error); + expect(actorsCreateMock).toHaveBeenCalledTimes(1); + expect(actorMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('says an earlier attempt may have created the Actor when the platform refuses the name on a retried POST', async () => { + // apify-client retries a POST that timed out or got a 5xx, and the platform may have saved the attempt. + actorsCreateMock.mockRejectedValue( + apiError(409, 'Some other Actor already has this name ("my-actor").', 'actor-name-not-unique', 2), + ); + + const result = await callTool({ autoBuild: true, files: [ACTOR_JSON] }); + + expectSoftFailInvalidInput(result); + expect(result.toolTelemetry).toEqual(expect.objectContaining({ failureHttpStatus: 409 })); + expect(result.structuredContent).toBeUndefined(); + expect(result.content).toStrictEqual([ + { + type: 'text', + text: + 'The platform refused the name my-actor as taken when the request was retried, so an earlier ' + + 'attempt of this call may have created the Actor my-actor. ' + + 'If it holds the files you sent, this call created it; get-actor-version reads its files.', + }, + ]); + expect(actorsCreateMock).toHaveBeenCalledTimes(1); + expect(actorMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('names get-actor-version on a retried POST only when the session has it', async () => { + actorsCreateMock.mockRejectedValue( + apiError(409, 'Some other Actor already has this name ("my-actor").', 'actor-name-not-unique', 2), + ); + const loadedToolNames = Object.values(HELPER_TOOLS).filter((name) => name !== HELPER_TOOLS.ACTOR_VERSION_GET); + const context = stubToolCallContext({ name: 'my-actor', files: [ACTOR_JSON] }, stubClient, { loadedToolNames }); + + const result = (await (createActor as HelperTool).call(context)) as CreateResult; + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe( + 'The platform refused the name my-actor as taken when the request was retried, so an earlier attempt ' + + 'of this call may have created the Actor my-actor. If it holds the files you sent, this call created it.', + ); + }); + + it('lets any other API error of a retried POST through unchanged', async () => { + const error = apiError(400, 'Invalid version number.', 'invalid-input', 2); + actorsCreateMock.mockRejectedValue(error); + + await expect(callTool({ files: [ACTOR_JSON] })).rejects.toBe(error); + }); + + describe('version number from .actor/actor.json', () => { + const actorJsonWith = (content: string) => ({ path: '.actor/actor.json', content }); + + it('takes the version of .actor/actor.json when versionNumber is not given', async () => { + const result = await callTool({ + autoBuild: true, + files: [actorJsonWith('{"actorSpecification": 1, "name": "my-actor", "version": "0.1"}')], + }); + + expect(getSentVersion().versionNumber).toBe('0.1'); + expect(result.structuredContent.versionNumber).toBe('0.1'); + expect(buildMock).toHaveBeenCalledWith('0.1', { useCache: true }); + }); + + it('takes the version of an .actor/actor.json that starts with a byte order mark, and sends the file as given', async () => { + const content = '\uFEFF{"actorSpecification": 1, "name": "my-actor", "version": "0.1"}'; + + await callTool({ files: [actorJsonWith(content)] }); + + expect(getSentVersion().versionNumber).toBe('0.1'); + expect(getSentVersion().sourceFiles[0].content).toBe(content); + }); + + it.each<[string, { path: string; content: string }[]]>([ + ['no .actor/actor.json', [MAIN_JS]], + ['an actor.json without a version', [ACTOR_JSON]], + ['an actor.json whose version is not a string', [actorJsonWith('{"version": 0.1}')]], + ['an actor.json with an empty version', [actorJsonWith('{"version": ""}')]], + ['an actor.json that is not JSON', [actorJsonWith('{"version": "0.1",')]], + ])('sends version 0.0 for %s', async (_, files) => { + const result = await callTool({ files }); + + expect(getSentVersion().versionNumber).toBe('0.0'); + expect(result.structuredContent.versionNumber).toBe('0.0'); + }); + + it.each([ + ['the version of .actor/actor.json', actorJsonWith('{"version": "0.1"}'), '0.1'], + ['version 0.0 when .actor/actor.json has none', ACTOR_JSON, '0.0'], + ])('sends %s for a null versionNumber', async (_, file, versionNumber) => { + const args = { name: 'my-actor', versionNumber: null, files: [file] }; + + expect((createActor as HelperTool).ajvValidate(args)).toBe(true); + await callTool(args); + + expect(getSentVersion().versionNumber).toBe(versionNumber); + }); + + it('sends versionNumber when .actor/actor.json has another version', async () => { + await callTool({ versionNumber: '0.2', files: [actorJsonWith('{"version": "0.1"}')] }); + + expect(getSentVersion().versionNumber).toBe('0.2'); + }); + + it('sends a version of .actor/actor.json that is not MAJOR.MINOR as is, for the API to refuse', async () => { + const error = apiError(400, 'Invalid version number.', 'invalid-input'); + actorsCreateMock.mockRejectedValue(error); + + await expect(callTool({ files: [actorJsonWith('{"version": "1.0.0"}')] })).rejects.toBe(error); + expect(getSentVersion().versionNumber).toBe('1.0.0'); + }); + }); + + describe('autoBuild', () => { + it('starts a build without waiting', async () => { + const result = await callTool({ autoBuild: true, files: [ACTOR_JSON, DOCKERFILE] }); + + expectSchemaConformingStructuredContent(result, createActorToolOutputSchema); + expect(actorMock).toHaveBeenCalledWith('actor-9'); + expect(buildMock).toHaveBeenCalledWith('0.0', { useCache: true }); + expect(result.structuredContent.build).toEqual( + expect.objectContaining({ id: 'build-1', buildNumber: '0.0.1', status: 'READY' }), + ); + expect(result.content).toHaveLength(2); + expect(JSON.parse(result.content[0].text)).toEqual(result.structuredContent); + }); + + it('adds the build Console link for an Apify Console session', async () => { + vi.mocked(getUserInfoCached).mockResolvedValue(mockUserInfo()); + + const result = (await (createActor as HelperTool).call({ + ...stubToolCallContext({ name: 'my-actor', autoBuild: true, files: [ACTOR_JSON] }, stubClient), + apifyToken: 'apify_ui_test', + })) as CreateResult; + + const consoleUrl = 'https://console.apify.com/actors/actor-9/builds/0.0.1'; + expectSchemaConformingStructuredContent(result, createActorToolOutputSchema); + expect(getUserInfoCached).toHaveBeenCalledWith('apify_ui_test', stubClient); + expect(result.structuredContent.build?.apifyConsoleUrl).toBe(consoleUrl); + expect(result.content).toHaveLength(3); + expect(JSON.parse(result.content[0].text)).toEqual(result.structuredContent); + expect(result.content[2].text).toBe(`Apify Console: ${consoleUrl}\n${VERBATIM_LINKS_NUDGE}`); + }); + + it('builds the version it created without waiting for the build', async () => { + buildMock.mockResolvedValue({ id: 'build-2', actId: 'actor-9', buildNumber: '1.2.1', status: 'RUNNING' }); + + const result = await callTool({ autoBuild: true, versionNumber: '1.2', files: [ACTOR_JSON] }); + + expect(actorMock.mock.calls).toEqual([['actor-9']]); + expect(buildMock.mock.calls).toEqual([['1.2', { useCache: true }]]); + expect(buildClientMock).not.toHaveBeenCalled(); + expect(result.structuredContent.build).toEqual( + expect.objectContaining({ id: 'build-2', buildNumber: '1.2.1', status: 'RUNNING' }), + ); + }); + + it('reports a build start that failed on the network as buildError, with the Actor still created', async () => { + buildMock.mockRejectedValue( + Object.assign(new Error('socket hang up'), { request: {}, config: {}, code: 'ECONNRESET' }), + ); + + const result = await callTool({ autoBuild: true, files: [ACTOR_JSON, DOCKERFILE] }); + + expectSchemaConformingStructuredContent(result, createActorToolOutputSchema); + expect(actorsCreateMock).toHaveBeenCalledTimes(1); + expect(result.structuredContent.actorId).toBe('actor-9'); + expect(result.structuredContent.buildError).toBe('socket hang up'); + expect(result.content[1].text).toBe( + 'Created the private Actor john/my-actor.\nThe build could not be started; runs use these files only after the version is built.', + ); + }); + + it('reports a build that failed to start with the Actor still created', async () => { + buildMock.mockRejectedValue(apiError(403, 'Build limit reached')); + + const result = await callTool({ autoBuild: true, files: [ACTOR_JSON, DOCKERFILE] }); + + expectSchemaConformingStructuredContent(result, createActorToolOutputSchema); + expect(result.structuredContent.actorId).toBe('actor-9'); + expect(result.structuredContent.buildError).toBe('Build limit reached'); + }); + }); +}); diff --git a/tests/unit/tools.get_actor_build_list.test.ts b/tests/unit/tools.get_actor_build_list.test.ts index f05fc7959..94f24c611 100644 --- a/tests/unit/tools.get_actor_build_list.test.ts +++ b/tests/unit/tools.get_actor_build_list.test.ts @@ -41,6 +41,11 @@ function mockPage(items: ReturnType[], overrides: Record, loadedToolNames?: readonly string[]) => { const context = stubToolCallContext(args, stubClient); if (loadedToolNames) context.loadedToolNames = loadedToolNames; @@ -154,18 +159,62 @@ describe('get-actor-build-list', () => { expect(content[1].text).toMatch(/^Your account has \d+ builds; showing 2 from offset 0\./); }); - it('returns a not-found error when the Actor does not exist', async () => { - listMock.mockRejectedValue(Object.assign(new Error('Actor was not found'), { statusCode: 404 })); + describe('actorId lookup', () => { + it('lists the builds of a bare name once, as the name of one of your own Actors', async () => { + listMock.mockResolvedValue(mockPage([mockBuild()])); + + const { content } = await callTool({ actorId: 'my-actor' }); + + expect(actorMock.mock.calls).toEqual([['~my-actor']]); + expect(listMock).toHaveBeenCalledTimes(1); + expect(content[1].text).toContain('Actor my-actor has 1 build; showing 1 from offset 0.'); + }); + + it('lists the builds of a bare value shaped like an ID as a name when no Actor has that ID', async () => { + listMock.mockRejectedValueOnce(buildNotFoundError()).mockResolvedValueOnce(mockPage([mockBuild()])); + + const { structuredContent } = await callTool({ actorId: 'myscraperversion2' }); + + expect(actorMock.mock.calls).toEqual([['myscraperversion2'], ['~myscraperversion2']]); + expect(structuredContent).toMatchObject({ total: 1, items: [{ id: 'build-1' }] }); + }); + + it('lists an empty page for an ID without retrying it as a name', async () => { + listMock.mockResolvedValue(mockPage([])); - const result = await (getActorBuildList as HelperTool).call( - stubToolCallContext({ actorId: 'missing' }, stubClient), + await callTool({ actorId: 'qGXMy0NAkWsIIb9LZ' }); + + expect(actorMock.mock.calls).toEqual([['qGXMy0NAkWsIIb9LZ']]); + }); + + it.each(['qGXMy0NAkWsIIb9LZ', 'john/my-actor', 'john~my-actor'])( + 'lists the builds of %s once, as given', + async (actorId) => { + listMock.mockResolvedValue(mockPage([mockBuild()])); + + await callTool({ actorId }); + + expect(actorMock.mock.calls).toEqual([[actorId]]); + }, ); - const { content, structuredContent } = result as TextToolResult & { structuredContent?: unknown }; - expectSoftFailInvalidInput(result); - expect(actorMock).toHaveBeenCalledWith('missing'); - expect(content[0].text).toBe("Actor 'missing' not found."); - expect(structuredContent).toBeUndefined(); + it.each([ + ['a bare name', 'missing', ['~missing']], + ['a bare value shaped like an ID', 'myscraperversion2', ['myscraperversion2', '~myscraperversion2']], + ['a full name', 'john/missing', ['john/missing']], + ])('returns a not-found error for %s that matches no Actor', async (_, actorId, lookups) => { + listMock.mockRejectedValue(buildNotFoundError()); + + const result = await (getActorBuildList as HelperTool).call(stubToolCallContext({ actorId }, stubClient)); + const { content, structuredContent } = result as TextToolResult & { structuredContent?: unknown }; + + expectSoftFailInvalidInput(result); + expect(actorMock.mock.calls).toEqual(lookups.map((lookup) => [lookup])); + expect(content[0].text).toBe( + `Actor '${actorId}' not found. The actorId argument takes an ID, a name (resolved against your own Actors), or a full name as username/name.`, + ); + expect(structuredContent).toBeUndefined(); + }); }); it('rethrows non-404 errors from the build list', async () => { diff --git a/tests/unit/tools.get_actor_run_list.test.ts b/tests/unit/tools.get_actor_run_list.test.ts index 1623eb1a8..77613e20a 100644 --- a/tests/unit/tools.get_actor_run_list.test.ts +++ b/tests/unit/tools.get_actor_run_list.test.ts @@ -40,6 +40,11 @@ const MOCK_RUNS = { ], }; +/** What the list endpoint rejects with when no Actor matches. */ +function buildNotFoundError() { + return Object.assign(new Error('Actor was not found'), { statusCode: 404 }); +} + describe('get-actor-run-list', () => { beforeEach(() => { vi.clearAllMocks(); @@ -94,17 +99,65 @@ describe('get-actor-run-list', () => { expectSchemaConformingStructuredContent(result, actorRunListOutputSchema); }); - it('returns isError with a not-found message when the Actor does not exist', async () => { - actorListMock.mockRejectedValue(Object.assign(new Error('Actor was not found'), { statusCode: 404 })); + describe('actorId lookup', () => { + const callTool = async (args: Record) => + (await (getActorRunList as HelperTool).call(stubToolCallContext(args, stubClient))) as TextToolResult; - const result = await (getActorRunList as HelperTool).call( - stubToolCallContext({ actorId: 'missing' }, stubClient), + it('lists the runs of a bare name once, as the name of one of your own Actors', async () => { + actorListMock.mockResolvedValue(MOCK_RUNS); + + const { structuredContent } = await callTool({ actorId: 'my-actor' }); + + expect(actorMock.mock.calls).toEqual([['~my-actor']]); + expect(actorListMock).toHaveBeenCalledTimes(1); + expect(structuredContent).toEqual(MOCK_RUNS); + }); + + it('lists the runs of a bare value shaped like an ID as a name when no Actor has that ID', async () => { + actorListMock.mockRejectedValueOnce(buildNotFoundError()).mockResolvedValueOnce(MOCK_RUNS); + + const { structuredContent } = await callTool({ actorId: 'myscraperversion2' }); + + expect(actorMock.mock.calls).toEqual([['myscraperversion2'], ['~myscraperversion2']]); + expect(structuredContent).toEqual(MOCK_RUNS); + }); + + it('lists an empty page for an ID without retrying it as a name', async () => { + actorListMock.mockResolvedValue({ ...MOCK_RUNS, total: 0, count: 0, items: [] }); + + await callTool({ actorId: 'qGXMy0NAkWsIIb9LZ' }); + + expect(actorMock.mock.calls).toEqual([['qGXMy0NAkWsIIb9LZ']]); + }); + + it.each(['qGXMy0NAkWsIIb9LZ', 'john/my-actor', 'john~my-actor'])( + 'lists the runs of %s once, as given', + async (actorId) => { + actorListMock.mockResolvedValue(MOCK_RUNS); + + await callTool({ actorId }); + + expect(actorMock.mock.calls).toEqual([[actorId]]); + }, ); - const { content, structuredContent } = result as TextToolResult & { structuredContent?: unknown }; - expectSoftFailInvalidInput(result); - expect(structuredContent).toBeUndefined(); - expect(content[0].text).toContain("Actor 'missing' not found"); + it.each([ + ['a bare name', 'missing', ['~missing']], + ['a bare value shaped like an ID', 'myscraperversion2', ['myscraperversion2', '~myscraperversion2']], + ['a full name', 'john/missing', ['john/missing']], + ])('returns isError with a not-found message for %s that matches no Actor', async (_, actorId, lookups) => { + actorListMock.mockRejectedValue(buildNotFoundError()); + + const result = await (getActorRunList as HelperTool).call(stubToolCallContext({ actorId }, stubClient)); + const { content, structuredContent } = result as TextToolResult & { structuredContent?: unknown }; + + expectSoftFailInvalidInput(result); + expect(actorMock.mock.calls).toEqual(lookups.map((lookup) => [lookup])); + expect(structuredContent).toBeUndefined(); + expect(content[0].text).toBe( + `Actor '${actorId}' not found. The actorId argument takes an ID, a name (resolved against your own Actors), or a full name as username/name.`, + ); + }); }); it('rethrows non-404 errors from the Actor run list', async () => { diff --git a/tests/unit/tools.get_actor_version.test.ts b/tests/unit/tools.get_actor_version.test.ts index 1c65148c8..4a8c5a571 100644 --- a/tests/unit/tools.get_actor_version.test.ts +++ b/tests/unit/tools.get_actor_version.test.ts @@ -6,9 +6,9 @@ import { beforeEach, describe, expect, it, vi } from 'vitest'; import { HELPER_TOOLS, MAX_INLINE_BYTES } from '../../src/const.js'; import { getCategoryTools, toolCategoriesEnabledByDefault } from '../../src/tools/index.js'; -import { getActorVersion } from '../../src/tools/source/get_actor_version.js'; -import { buildFilesRevision, compareSourcePaths } from '../../src/tools/source/source_files.js'; import { getActorVersionToolOutputSchema } from '../../src/tools/structured_output_schemas.js'; +import { getActorVersion } from '../../src/tools/versions/get_actor_version.js'; +import { buildFilesRevision, compareSourcePaths } from '../../src/tools/versions/source_files.js'; import type { HelperTool, InternalToolArgs } from '../../src/types.js'; import { expectSchemaConformingStructuredContent, @@ -180,9 +180,13 @@ describe('get-actor-version', () => { } }); - it('is served in the source category, which is not enabled by default', () => { - expect(getCategoryTools().source.map((tool) => tool.name)).toEqual([HELPER_TOOLS.ACTOR_VERSION_GET]); - expect(toolCategoriesEnabledByDefault).not.toContain('source'); + it('is served in the versions category, which is not enabled by default', () => { + expect(getCategoryTools().versions.map((tool) => tool.name)).toEqual([ + HELPER_TOOLS.ACTOR_VERSION_GET, + HELPER_TOOLS.ACTOR_CREATE, + HELPER_TOOLS.ACTOR_VERSION_UPDATE, + ]); + expect(toolCategoriesEnabledByDefault).not.toContain('versions'); }); describe('SOURCE_FILES', () => { @@ -217,7 +221,7 @@ describe('get-actor-version', () => { }, ); - it('sorts the manifest in plain string order and computes the revision over sorted path and hash lines', async () => { + it('sorts the manifest in plain string order and computes the revision over the sorted path and hash pairs', async () => { // Locale order would put package.json before README.md; plain order puts the emoji, a UTF-16 surrogate pair, // before the fullwidth z. const names = ['\u{1F600}.txt', 'package.json', 'src/main.js', '\uFF5A.txt', 'README.md', 'Dockerfile']; @@ -234,8 +238,8 @@ describe('get-actor-version', () => { const { structuredContent } = await callTool({}); expect(structuredContent.files.map(({ path }) => path)).toEqual(sortedNames); - const lines = sortedNames.map((name) => `${name}\0${sha256Prefix(`${name}\n`)}\n`).join(''); - expect(structuredContent.revision).toBe(sha256Prefix(lines)); + const pairs = sortedNames.map((name) => [name, sha256Prefix(`${name}\n`)]); + expect(structuredContent.revision).toBe(sha256Prefix(JSON.stringify(pairs))); }); it('gives the same revision whatever order the files are stored in, and a new one when a file changes', async () => { @@ -251,6 +255,23 @@ describe('get-actor-version', () => { expect(changed.structuredContent.revision).not.toBe(first.structuredContent.revision); }); + it('gives a version whose stored name holds NUL and a line break a revision of its own', async () => { + // Joined as path, NUL, hash, and a line break per file, the one file read the same as the two. + mockVersionRead({ + sourceFiles: [ + { name: 'a', format: 'TEXT', content: 'A' }, + { name: 'b', format: 'TEXT', content: 'B' }, + ], + }); + const twoFiles = await callTool({}); + const name = `a\0${sha256Prefix('A')}\nb`; + mockVersionRead({ sourceFiles: [{ name, format: 'TEXT', content: 'B' }] }); + const oneFile = await callTool({}); + + expect(oneFile.structuredContent.files).toEqual([{ path: name, sizeBytes: 1, hash: sha256Prefix('B') }]); + expect(oneFile.structuredContent.revision).not.toBe(twoFiles.structuredContent.revision); + }); + it('gives a new revision when a file is added, removed, or renamed', async () => { const first = await callTool({}); mockVersionRead({ @@ -598,7 +619,7 @@ describe('get-actor-version', () => { actorId: 'actor-1', fullName: 'john/my-actor', versionNumber: '0.1', - revision: sha256Prefix(''), + revision: sha256Prefix('[]'), files: [], contents: [], }); @@ -824,6 +845,55 @@ describe('get-actor-version', () => { expect(structuredContent.files.map(({ path }) => path)).toEqual(['src/main.js']); }); + it('reads the only version for a null versionNumber, which the input check turns into an empty string', async () => { + const args = { actor: 'john/my-actor', versionNumber: null }; + + expect((getActorVersion as HelperTool).ajvValidate(args)).toBe(true); + const { structuredContent } = await callTool(args); + + expect(structuredContent.versionNumber).toBe('0.1'); + }); + + it.each(['qGXMy0NAkWsIIb9LZ', 'john~my-actor'])( + 'names the Actor by the full name in its document when given as %s', + async (actor) => { + const result = await callTool({ actor }); + + expect(actorMock.mock.calls).toEqual([[actor]]); + expect(result.structuredContent.fullName).toBe('john/my-actor'); + expect(result.content[1].text).toBe('Read version 0.1 of john/my-actor.'); + }, + ); + + it('looks up a bare name once, as the name of one of your own Actors', async () => { + const result = await callTool({ actor: 'my-actor' }); + + expect(actorMock.mock.calls).toEqual([['~my-actor']]); + expect(result.structuredContent.fullName).toBe('john/my-actor'); + }); + + it('looks up a bare value shaped like an ID as a name when no Actor has that ID', async () => { + actorGetMock + .mockResolvedValueOnce(undefined) + .mockResolvedValueOnce({ ...mockActor(), name: 'myscraperversion2' }); + + const result = await callTool({ actor: 'myscraperversion2' }); + + expect(actorMock.mock.calls).toEqual([['myscraperversion2'], ['~myscraperversion2']]); + expect(result.structuredContent.fullName).toBe('john/myscraperversion2'); + }); + + it.each(['john/missing', 'john~missing', '~missing'])('looks up %s once', async (actor) => { + actorGetMock.mockResolvedValue(undefined); + + const text = await callToolExpectingUserError({ actor }); + + expect(actorMock.mock.calls).toEqual([[actor]]); + expect(text).toBe( + `Actor '${actor}' not found. The actor argument takes an ID, a name (resolved against your own Actors), or a full name as username/name.`, + ); + }); + it('refuses a version the Actor does not have', async () => { const text = await callToolExpectingUserError({ versionNumber: '9.9' }); @@ -831,7 +901,6 @@ describe('get-actor-version', () => { }); it.each([ - ['a bare name the API does not find', 'my-actor', undefined], [ 'a run reached by extra path segments', 'john/my-actor/runs/last', @@ -850,14 +919,14 @@ describe('get-actor-version', () => { 'john/my-actor/runs/last/key-value-store', { id: 'kvs-1', userId: 'user-secret', username: 'john' }, ], - ])('reports %s as a missing Actor, and that a bare name is not enough', async (_, actor, document) => { + ])('reports %s as a missing Actor', async (_, actor, document) => { actorGetMock.mockResolvedValue(document); const text = await callToolExpectingUserError({ actor }); - expect(actorMock).toHaveBeenCalledWith(actor); + expect(actorMock.mock.calls).toEqual([[actor]]); expect(text).toBe( - `Actor '${actor}' not found. Give its ID or its full name, username/name; a name without the username is not enough.`, + `Actor '${actor}' not found. The actor argument takes an ID, a name (resolved against your own Actors), or a full name as username/name.`, ); }); @@ -1009,12 +1078,33 @@ describe('compareSourcePaths()', () => { }); describe('buildFilesRevision()', () => { + it.each<[string, { path: string; hash: string }[], { path: string; hash: string }[]]>([ + [ + 'a path that holds NUL, a hash, and a line break', + [ + { path: 'a', hash: 'hash-a' }, + { path: 'b', hash: 'hash-b' }, + ], + [{ path: 'a\0hash-a\nb', hash: 'hash-b' }], + ], + [ + 'a path that holds a quote and a comma', + [ + { path: 'a', hash: 'hash-a' }, + { path: 'b', hash: 'hash-b' }, + ], + [{ path: 'a","hash-a"],["b', hash: 'hash-b' }], + ], + ])('tells apart two file sets when %s', (_, files, otherFiles) => { + expect(buildFilesRevision(otherFiles)).not.toBe(buildFilesRevision(files)); + }); + it('sorts the files by path itself, so callers can pass them in any order', () => { const revision = buildFilesRevision([ { path: 'b', hash: 'hash-b' }, { path: 'a', hash: 'hash-a' }, ]); - expect(revision).toBe(sha256Prefix('a\0hash-a\nb\0hash-b\n')); + expect(revision).toBe(sha256Prefix('[["a","hash-a"],["b","hash-b"]]')); }); }); diff --git a/tests/unit/tools.mode_contract.test.ts b/tests/unit/tools.mode_contract.test.ts index ff580f100..9f06c6416 100644 --- a/tests/unit/tools.mode_contract.test.ts +++ b/tests/unit/tools.mode_contract.test.ts @@ -139,10 +139,14 @@ describe('getCategoryTools mode contract (tool-mode separation)', () => { expect(toolNames(appsCategories.builds)).toEqual(expected); }); - it('has correct tools in source category (both modes)', () => { - const expected = [HELPER_TOOLS.ACTOR_VERSION_GET]; - expect(toolNames(defaultCategories.source)).toEqual(expected); - expect(toolNames(appsCategories.source)).toEqual(expected); + it('has correct tools in versions category (both modes)', () => { + const expected = [ + HELPER_TOOLS.ACTOR_VERSION_GET, + HELPER_TOOLS.ACTOR_CREATE, + HELPER_TOOLS.ACTOR_VERSION_UPDATE, + ]; + expect(toolNames(defaultCategories.versions)).toEqual(expected); + expect(toolNames(appsCategories.versions)).toEqual(expected); }); it('has correct tools in api category (both modes)', () => { @@ -520,7 +524,7 @@ describe('tool descriptions never name a tool absent from the session', () => { { label: "tools: ['storage']", input: { tools: ['storage'] }, actorTools: [] }, { label: "tools: ['actors']", input: { tools: ['actors'] }, actorTools: [] }, { label: "tools: ['schedules']", input: { tools: ['schedules'] }, actorTools: [] }, - { label: "tools: ['source']", input: { tools: ['source'] }, actorTools: [] }, + { label: "tools: ['versions']", input: { tools: ['versions'] }, actorTools: [] }, { label: "tools: ['api']", input: { tools: ['api'] }, actorTools: [] }, ...ALL_TOOL_NAMES.map((name) => ({ label: `tools: ['${name}']`, diff --git a/tests/unit/tools.update_actor_version.test.ts b/tests/unit/tools.update_actor_version.test.ts new file mode 100644 index 000000000..f99815ee2 --- /dev/null +++ b/tests/unit/tools.update_actor_version.test.ts @@ -0,0 +1,2701 @@ +import { createHash } from 'node:crypto'; + +import { ApifyApiError } from 'apify-client'; +import type { AxiosResponse } from 'axios'; +import { beforeEach, describe, expect, it, vi } from 'vitest'; + +import { FAILURE_CATEGORY, HELPER_TOOLS, TOOL_STATUS } from '../../src/const.js'; +import { prepareToolCallContext } from '../../src/payments/helpers.js'; +import { updateActorVersionToolOutputSchema } from '../../src/tools/structured_output_schemas.js'; +import { getActorVersion } from '../../src/tools/versions/get_actor_version.js'; +import { buildFilesRevision } from '../../src/tools/versions/source_files.js'; +import { updateActorVersion } from '../../src/tools/versions/update_actor_version.js'; +import type { HelperTool, InternalToolArgs } from '../../src/types.js'; +import { VERBATIM_LINKS_NUDGE } from '../../src/utils/console_link.js'; +import { getUserInfoCached } from '../../src/utils/userid_cache.js'; +import { + expectSchemaConformingStructuredContent, + expectSoftFailInvalidInput, + mockUserInfo, + stubToolCallContext, + type TextToolResult, + type ToolTelemetrySnapshot, +} from './helpers/tool_context.js'; + +vi.mock('../../src/utils/userid_cache.js', () => ({ + getUserInfoCached: vi.fn(), +})); + +const actorGetMock = vi.fn(); +const versionUpdateMock = vi.fn(); +const buildMock = vi.fn(); +const versionMock = vi.fn(() => ({ update: versionUpdateMock })); +const actorMock = vi.fn(() => ({ get: actorGetMock, version: versionMock, build: buildMock })); +// The build client, which only waiting for a build uses. +const buildClientMock = vi.fn(); + +const stubClient = { actor: actorMock, build: buildClientMock } as unknown as InternalToolArgs['apifyClient']; + +const ACTOR_JSON = { name: '.actor/actor.json', format: 'TEXT', content: '{"actorSpecification": 1}' }; +const MAIN_JS = { name: 'src/main.js', format: 'TEXT', content: 'const a = 1;\nconsole.log(a);\nexport {};\n' }; +const LOGO_BYTES = Buffer.from([137, 80, 78, 71, 0, 255]); +const LOGO = { name: 'assets/logo.png', format: 'BASE64', content: LOGO_BYTES.toString('base64') }; +const FOLDER = { name: 'storage', folder: true }; +const EMPTY_INIT = { name: 'src/__init__.py', format: 'TEXT', content: '' }; +// Stored without content, which the build worker reads as empty. +const NO_CONTENT = { name: 'src/blank.js', format: 'TEXT' }; + +type UpdateOutput = { + revision: string; + changed: boolean; + changes: { path: string; action: string; hash?: string }[]; + warnings?: string[]; + build?: Record; + buildError?: string; +}; + +type UpdateResult = TextToolResult & { structuredContent: UpdateOutput; toolTelemetry?: ToolTelemetrySnapshot }; + +function sha256Prefix(data: Buffer | string): string { + return createHash('sha256').update(data).digest('hex').slice(0, 16); +} + +const MAIN_JS_HASH = sha256Prefix(MAIN_JS.content); + +/** A SOURCE_FILES version with env vars, which a write must never send back. */ +function mockVersion(overrides: Record = {}) { + return { + versionNumber: '0.1', + buildTag: 'latest', + sourceType: 'SOURCE_FILES', + envVars: [{ name: 'API_KEY', value: 'secret-value', isSecret: true }], + sourceFiles: [ACTOR_JSON, MAIN_JS, LOGO, FOLDER], + ...overrides, + }; +} + +/** The Actor GET, which carries the versions with their files. */ +function mockVersionRead(overrides: Record = {}) { + actorGetMock.mockResolvedValue({ + id: 'actor-1', + name: 'my-actor', + username: 'john', + versions: [mockVersion(overrides)], + }); +} + +function mockFiles(...sourceFiles: Record[]) { + mockVersionRead({ sourceFiles }); +} + +/** `attempt` is the attempt of the request apify-client gave up on; above 1, it retried the request. */ +function apiError(status: number, message: string, type = 'some-error', attempt = 1): ApifyApiError { + return new ApifyApiError({ data: { error: { type, message } }, status } as AxiosResponse, attempt); +} + +async function callTool(args: Record, signal?: AbortSignal): Promise { + const context = stubToolCallContext({ actor: 'john/my-actor', ...args }, stubClient); + const withSignal = signal === undefined ? context : { ...context, signal }; + return (await (updateActorVersion as HelperTool).call(withSignal)) as UpdateResult; +} + +/** A call from an Apify Console session, whose UI token gets Console links. */ +async function callToolInConsole(args: Record, signal?: AbortSignal): Promise { + const context = stubToolCallContext({ actor: 'john/my-actor', ...args }, stubClient); + const withSignal = signal === undefined ? context : { ...context, signal }; + return (await (updateActorVersion as HelperTool).call({ + ...withSignal, + apifyToken: 'apify_ui_test', + })) as UpdateResult; +} + +async function callToolExpectingUserError(args: Record) { + const result = await callTool(args); + expectSoftFailInvalidInput(result); + expect(versionUpdateMock).not.toHaveBeenCalled(); + return result.content[0].text; +} + +/** The files of the one version PUT the call sent. */ +function getPutFiles(): Record[] { + expect(versionUpdateMock).toHaveBeenCalledTimes(1); + return (versionUpdateMock.mock.calls[0][0] as { sourceFiles: Record[] }).sourceFiles; +} + +type VersionRead = { + revision: string; + files: { path: string; sizeBytes: number; hash: string }[]; + contents: { path: string; content: string; encoding: string }[]; +}; + +async function readVersion(args: Record = {}): Promise { + const result = (await (getActorVersion as HelperTool).call( + stubToolCallContext({ actor: 'john/my-actor', ...args }, stubClient), + )) as { structuredContent: VersionRead }; + return result.structuredContent; +} + +async function readRevision(): Promise { + return (await readVersion()).revision; +} + +const write = (path: string, content: string, extra: Record = {}) => ({ + type: 'write', + path, + content, + ...extra, +}); + +const edit = (path: string, ...edits: { oldText: string; newText: string }[]) => ({ type: 'edit', path, edits }); + +const remove = (path: string, expectedHash: string) => ({ type: 'delete', path, expectedHash }); + +/** A precondition failure message, up to the detail that follows the reason. */ +const failedWith = (index: number, type: string, path: string, reason: string) => + `Nothing was written: operations[${index}] (${type} ${path}) failed with ${reason}.`; + +describe('update-actor-version', () => { + beforeEach(() => { + vi.clearAllMocks(); + mockVersionRead(); + versionUpdateMock.mockResolvedValue({}); + buildMock.mockResolvedValue({ + id: 'build-1', + actId: 'actor-1', + buildNumber: '0.1.5', + status: 'READY', + startedAt: new Date('2026-09-01T10:00:00.000Z'), + }); + }); + + it('is a destructive, non-idempotent, open-world tool without payment, since autoBuild starts a build', () => { + expect(updateActorVersion.name).toBe(HELPER_TOOLS.ACTOR_VERSION_UPDATE); + expect(updateActorVersion.annotations).toEqual({ + title: 'Update Actor version', + readOnlyHint: false, + destructiveHint: true, + idempotentHint: false, + openWorldHint: true, + }); + expect((updateActorVersion as HelperTool).paymentRequired).toBeUndefined(); + }); + + it('names the reason codes of a failed check in its description without asking for a retry', () => { + expect(updateActorVersion.description).toContain( + 'one of FILE_EXISTS, FILE_NOT_FOUND, HASH_MISMATCH, NO_MATCH, MULTIPLE_MATCHES, NOT_TEXT, or REVISION_MISMATCH.', + ); + expect(updateActorVersion.description).not.toMatch(/retry/i); + }); + + it('reads the Actor once and sends one PUT with only sourceType and sourceFiles', async () => { + const result = await callTool({ operations: [write('src/util.js', 'export const b = 2;\n')] }); + + expectSchemaConformingStructuredContent(result, updateActorVersionToolOutputSchema); + expect(actorGetMock).toHaveBeenCalledTimes(1); + expect(actorMock).toHaveBeenNthCalledWith(1, 'john/my-actor'); + expect(actorMock).toHaveBeenNthCalledWith(2, 'actor-1'); + expect(versionMock).toHaveBeenCalledWith('0.1'); + // Folder entries go back as stored, then the stored files in path order; new files follow in the order written. + expect(versionUpdateMock).toHaveBeenCalledWith({ + sourceType: 'SOURCE_FILES', + sourceFiles: [ + FOLDER, + ACTOR_JSON, + LOGO, + MAIN_JS, + { name: 'src/util.js', format: 'TEXT', content: 'export const b = 2;\n' }, + ], + }); + expect(result.structuredContent).toEqual({ + revision: expect.stringMatching(/^[0-9a-f]{16}$/), + changed: true, + changes: [{ path: 'src/util.js', action: 'created', hash: sha256Prefix('export const b = 2;\n') }], + }); + expect(result.content).toHaveLength(2); + expect(JSON.parse(result.content[0].text)).toEqual(result.structuredContent); + expect(result.content[1].text).toBe( + 'Updated version 0.1 of john/my-actor.\nNo build was started; runs use these files only after the version is built.', + ); + expect(buildMock).not.toHaveBeenCalled(); + }); + + describe('input schema', () => { + const validate = (args: Record) => (updateActorVersion as HelperTool).ajvValidate(args); + const fullOperation = { + type: 'edit', + path: 'a.js', + content: 'x', + encoding: 'utf8', + expectedHash: MAIN_JS_HASH, + edits: [{ oldText: 'a', newText: 'b' }], + }; + + it('requires only actor and operations', () => { + expect((updateActorVersion as HelperTool).inputSchema.required).toEqual(['actor', 'operations']); + }); + + it.each(['write', 'edit', 'delete'])( + 'keeps every field of an operation of type %s, the fields of the other types included, and strips unknown keys', + (type) => { + const args = { + actor: 'john/my-actor', + operations: [{ ...fullOperation, type, extra: 1 }], + expectedRevision: 'abc', + autoBuild: true, + unknown: 'x', + }; + + expect(validate(args)).toBe(true); + expect(args).toStrictEqual({ + actor: 'john/my-actor', + operations: [{ ...fullOperation, type }], + expectedRevision: 'abc', + autoBuild: true, + }); + }, + ); + + it.each<[string, Record]>([ + ['no actor', { operations: [] }], + ['an empty actor', { actor: '', operations: [] }], + ['no operations', { actor: 'john/my-actor' }], + ['an unknown operation type', { actor: 'a', operations: [{ type: 'rename', path: 'a.js' }] }], + ['an operation without a type', { actor: 'a', operations: [{ path: 'a.js', content: 'x' }] }], + ['an empty path', { actor: 'a', operations: [{ type: 'write', path: '', content: 'x' }] }], + ['an operation without a path', { actor: 'a', operations: [{ type: 'write', content: 'x' }] }], + [ + 'an unknown encoding', + { actor: 'a', operations: [{ type: 'write', path: 'a', content: 'x', encoding: 'hex' }] }, + ], + ['empty edits', { actor: 'a', operations: [{ type: 'edit', path: 'a.js', edits: [] }] }], + [ + 'an empty oldText', + { actor: 'a', operations: [{ type: 'edit', path: 'a.js', edits: [{ oldText: '', newText: 'b' }] }] }, + ], + [ + 'an edit without newText', + { actor: 'a', operations: [{ type: 'edit', path: 'a.js', edits: [{ oldText: 'a' }] }] }, + ], + ])('rejects %s', (_, args) => { + expect(validate(args)).toBe(false); + }); + + it('accepts no operations and an empty newText', () => { + expect(validate({ actor: 'a', operations: [] })).toBe(true); + expect( + validate({ + actor: 'a', + operations: [{ type: 'edit', path: 'a.js', edits: [{ oldText: 'a', newText: '' }] }], + }), + ).toBe(true); + }); + }); + + describe('limits', () => { + const writes = (count: number) => Array.from({ length: count }, (_, index) => write(`src/f${index}.js`, 'x')); + // Each line matches only its own oldText, so every edit applies. + const LINES_JS = { + name: 'src/lines.js', + format: 'TEXT', + content: Array.from({ length: 21 }, (_, index) => `line ${index};\n`).join(''), + }; + const renameLines = (from: number, to: number) => + edit( + 'src/lines.js', + ...Array.from({ length: to - from }, (_, offset) => ({ + oldText: `line ${from + offset};`, + newText: `row ${from + offset};`, + })), + ); + + it('takes 100 operations', async () => { + const operations = writes(100); + + expect((updateActorVersion as HelperTool).ajvValidate({ actor: 'a', operations })).toBe(true); + await callTool({ operations }); + expect(getPutFiles()).toHaveLength(104); + }); + + it('refuses 101 operations without reading or writing the version', async () => { + const operations = writes(101); + + expect((updateActorVersion as HelperTool).ajvValidate({ actor: 'a', operations })).toBe(false); + await expect(callTool({ operations })).rejects.toThrow(); + expect(actorGetMock).not.toHaveBeenCalled(); + expect(versionUpdateMock).not.toHaveBeenCalled(); + }); + + it('takes 20 edits in total across the edit operations', async () => { + mockFiles(LINES_JS); + + await callTool({ operations: [renameLines(0, 10), renameLines(10, 20)] }); + + const [saved] = getPutFiles() as { content: string }[]; + expect(saved.content).toContain('row 19;\nline 20;\n'); + expect(saved.content).not.toContain('line 19;'); + }); + + it('refuses 21 edits in total across the edit operations without reading or writing the version', async () => { + mockFiles(LINES_JS); + + const text = await callToolExpectingUserError({ + autoBuild: true, + operations: [renameLines(0, 10), write('src/new.js', 'x'), renameLines(10, 21)], + }); + + expect(text).toBe( + 'Nothing was written: the operations have 21 edits in total, and one call takes at most 20.', + ); + expect(actorGetMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('searches a file of almost 3 MiB for a 10,001-character oldText within a second (NO_MATCH)', async () => { + mockFiles({ name: 'src/big.txt', format: 'TEXT', content: 'a'.repeat(3 * 1024 * 1024 - 20_000) }); + const oldText = `${'a'.repeat(5000)}b${'a'.repeat(5000)}`; + + const startedAt = performance.now(); + const text = await callToolExpectingUserError({ + operations: [edit('src/big.txt', { oldText, newText: '' })], + }); + + expect(performance.now() - startedAt).toBeLessThan(1000); + expect(text).toBe( + `${failedWith(0, 'edit', 'src/big.txt', 'NO_MATCH')} oldText of edits[0] is not in the file.`, + ); + }); + + it('applies 20 edits with an oldText over 250 characters to a file of almost 3 MiB within 6 seconds', async () => { + // Each oldText is 300 a's and its own suffix, so it matches once, and the search reads the whole file: over + // the runs of a it keeps 300 characters matched and steps back one on every character. + const oldTexts = Array.from({ length: 20 }, (_, index) => `${'a'.repeat(300)}#${index};`); + const tail = oldTexts.join(''); + const content = `${'a'.repeat(3 * 1024 * 1024 - 20_000 - tail.length)}${tail}`; + mockFiles({ name: 'src/big.txt', format: 'TEXT', content }); + + const startedAt = performance.now(); + await callTool({ + operations: oldTexts.map((oldText) => + edit('src/big.txt', { oldText, newText: oldText.replace('#', '@') }), + ), + }); + + // Bounds the worst-case search times the cap; the 21-edit test pins the cap itself. About 0.5 s locally, and + // CI runs this file about 4.5x slower. + expect(performance.now() - startedAt).toBeLessThan(6000); + const [saved] = getPutFiles() as { content: string }[]; + expect(saved.content.slice(-tail.length)).toBe(tail.replaceAll('#', '@')); + }); + + describe('length of an edited file', () => { + const MAX_LENGTH = 3 * 1024 * 1024; + // 91 UTF-16 code units under the limit. + const NEAR_LIMIT = { + name: 'src/big.txt', + format: 'TEXT', + content: `${'a'.repeat(MAX_LENGTH - 100)}#0;#1;#2;`, + }; + const grow = (marker: string, by: number) => ({ oldText: marker, newText: `${marker}${'x'.repeat(by)}` }); + + it('takes edits that grow a file to exactly 3 MiB of UTF-16 code units', async () => { + mockFiles(NEAR_LIMIT); + + await callTool({ operations: [edit('src/big.txt', grow('#0;', 50), grow('#1;', 41))] }); + + const [saved] = getPutFiles() as { content: string }[]; + expect(saved.content).toHaveLength(MAX_LENGTH); + }); + + it('refuses the edit that grows a file past 3 MiB of UTF-16 code units, before searching for the next one', async () => { + mockFiles(NEAR_LIMIT); + + const text = await callToolExpectingUserError({ + operations: [ + edit('src/big.txt', grow('#0;', 50)), + // #9; is not in the file, so a search for it would fail with NO_MATCH. + edit('src/big.txt', grow('#1;', 42), grow('#9;', 1)), + ], + }); + + expect(text).toBe( + 'Nothing was written: edits[0] of operations[1] (edit src/big.txt) can make the file longer than ' + + '3145728 UTF-16 code units, the limit for an edited file.', + ); + }); + + it('refuses an edit whose newText grows a file past the limit before searching for its oldText', async () => { + mockFiles(NEAR_LIMIT); + + // #9; is not in the file, so a search for it would fail with NO_MATCH. + const text = await callToolExpectingUserError({ operations: [edit('src/big.txt', grow('#9;', 92))] }); + + expect(text).toBe( + 'Nothing was written: edits[0] of operations[0] (edit src/big.txt) can make the file longer than ' + + '3145728 UTF-16 code units, the limit for an edited file.', + ); + }); + + describe('with CRLF line breaks', () => { + // 93 UTF-16 code units under the limit. + const CRLF_NEAR_LIMIT = { + name: 'src/big.txt', + format: 'TEXT', + content: `${'a'.repeat(MAX_LENGTH - 100)}\r\n#0;\r\n`, + }; + + it.each([ + ['LFs, each of which becomes CRLF', 'b\n'.repeat(32)], + ['CRLFs, which stay as they are', 'b\r\n'.repeat(32)], + ['no line break', 'b'.repeat(96)], + ])('takes an edit that grows the file to exactly 3 MiB with a newText of %s', async (_, newText) => { + mockFiles(CRLF_NEAR_LIMIT); + + await callTool({ operations: [edit('src/big.txt', { oldText: '#0;', newText })] }); + + const [saved] = getPutFiles() as { content: string }[]; + expect(saved.content).toHaveLength(MAX_LENGTH); + }); + + it('refuses an edit whose newText grows the file past the limit once its LFs become CRLF', async () => { + mockFiles(CRLF_NEAR_LIMIT); + + // 65 code units, and 97 once each LF becomes CRLF. + const text = await callToolExpectingUserError({ + operations: [edit('src/big.txt', { oldText: '#0;', newText: `${'b\n'.repeat(32)}b` })], + }); + + expect(text).toBe( + 'Nothing was written: edits[0] of operations[0] (edit src/big.txt) can make the file longer than ' + + '3145728 UTF-16 code units, the limit for an edited file.', + ); + }); + }); + }); + }); + + describe('logged arguments', () => { + const SECRET = 'synthetic-secret-4f2a'; + const getLoggedArgs = (args: Record) => + prepareToolCallContext({ provider: undefined, tool: updateActorVersion, args, apifyToken: 'test-token' }) + .toolArgsRedacted; + + it('keep the paths, operation types, and hashes, and replace content and edit texts with a marker', () => { + const args = { + actor: 'john/my-actor', + versionNumber: '0.1', + expectedRevision: 'abc', + autoBuild: true, + operations: [ + write('src/config.js', `export const key = '${SECRET}';\n`, { encoding: 'utf8' }), + { + ...edit('src/main.js', { oldText: `'${SECRET}'`, newText: `process.env.KEY ?? '${SECRET}'` }), + expectedHash: MAIN_JS_HASH, + }, + remove('src/old.js', MAIN_JS_HASH), + ], + undeclared: SECRET, + }; + const argsBefore = structuredClone(args); + + const logged = getLoggedArgs(args); + + expect(JSON.stringify(logged)).not.toContain(SECRET); + expect(logged).toStrictEqual({ + actor: 'john/my-actor', + versionNumber: '0.1', + expectedRevision: 'abc', + autoBuild: true, + operations: [ + { type: 'write', path: 'src/config.js', encoding: 'utf8', content: '[REDACTED]' }, + { + type: 'edit', + path: 'src/main.js', + expectedHash: MAIN_JS_HASH, + edits: [{ oldText: '[REDACTED]', newText: '[REDACTED]' }], + }, + { type: 'delete', path: 'src/old.js', expectedHash: MAIN_JS_HASH }, + ], + }); + expect(args).toStrictEqual(argsBefore); + }); + + it.each<[string, Record]>([ + ['no arguments', {}], + ['operations that are not a list', { operations: SECRET }], + ['operations that are not objects', { operations: [SECRET, null, 1, [SECRET]] }], + ['edits that are not a list', { operations: [{ type: 'edit', path: 'a.js', edits: SECRET }] }], + ['edits that are not objects', { operations: [{ type: 'edit', path: 'a.js', edits: [SECRET, null] }] }], + ['a path that is not a string', { operations: [{ type: 'write', path: { text: SECRET } }] }], + ['content that is not a string', { operations: [{ type: 'write', path: 'a.js', content: [SECRET] }] }], + ['an actor that is not a string', { actor: { name: SECRET }, operations: [] }], + ])('accept %s without throwing or logging their values', (_, args) => { + const argsBefore = structuredClone(args); + + expect(JSON.stringify(getLoggedArgs(args))).not.toContain(SECRET); + expect(args).toStrictEqual(argsBefore); + }); + }); + + it('reports the revision get-actor-version returns after the write', async () => { + const revisionBefore = await readRevision(); + const result = await callTool({ + expectedRevision: revisionBefore, + operations: [{ type: 'delete', path: 'src/main.js', expectedHash: MAIN_JS_HASH }], + }); + mockFiles(...getPutFiles()); + + expect(result.structuredContent.revision).not.toBe(revisionBefore); + expect(result.structuredContent.revision).toBe(await readRevision()); + }); + + it('returns the hashes and revision get-actor-version lists for the saved files', async () => { + const icon = Buffer.from([0, 1, 2, 255]); + const result = await callTool({ + operations: [ + edit('src/main.js', { oldText: 'const a = 1;', newText: 'const a = 2;' }), + write('assets/icon.png', icon.toString('base64')), + write('README.md', Buffer.from('héllo\n').toString('base64'), { encoding: 'base64' }), + remove('.actor/actor.json', sha256Prefix(ACTOR_JSON.content)), + ], + }); + mockFiles(...getPutFiles()); + const read = await readVersion({ paths: ['assets/icon.png', 'README.md'] }); + + expect(result.structuredContent.revision).toBe(read.revision); + const listedHashes = new Map(read.files.map(({ path, hash }) => [path, hash])); + for (const change of result.structuredContent.changes) { + expect(change.hash, change.path).toBe(listedHashes.get(change.path)); + } + expect(result.structuredContent.changes.map(({ path, action }) => `${action} ${path}`)).toEqual([ + 'deleted .actor/actor.json', + 'created README.md', + 'created assets/icon.png', + 'updated src/main.js', + ]); + // The base64 content decodes to exactly the bytes sent. + expect(read.contents).toEqual([ + { path: 'assets/icon.png', content: icon.toString('base64'), encoding: 'base64' }, + { path: 'README.md', content: 'héllo\n', encoding: 'utf8' }, + ]); + }); + + describe('files left alone', () => { + const readmeAsBase64 = { + name: 'README.md', + format: 'BASE64', + content: Buffer.from('# Hi\n').toString('base64'), + }; + // Stored without format, which the build worker reads as TEXT. + const noFormat = { name: 'src/z.js', content: 'z();\n' }; + const nestedFolder = { name: 'src/empty', folder: true }; + + it('sends every file it leaves alone back exactly as stored, before and after the changed one', async () => { + mockFiles( + MAIN_JS, + FOLDER, + readmeAsBase64, + noFormat, + EMPTY_INIT, + ACTOR_JSON, + NO_CONTENT, + LOGO, + nestedFolder, + ); + + const result = await callTool({ + operations: [edit('.actor/actor.json', { oldText: '1', newText: '2' })], + }); + + const actorJson = { name: '.actor/actor.json', format: 'TEXT', content: '{"actorSpecification": 2}' }; + expect(versionUpdateMock.mock.calls).toStrictEqual([ + [ + { + sourceType: 'SOURCE_FILES', + sourceFiles: [ + FOLDER, + nestedFolder, + actorJson, + readmeAsBase64, + LOGO, + EMPTY_INIT, + NO_CONTENT, + MAIN_JS, + noFormat, + ], + }, + ], + ]); + expect(result.structuredContent.changes).toEqual([ + { path: '.actor/actor.json', action: 'updated', hash: sha256Prefix(actorJson.content) }, + ]); + }); + + it('sends back an entry stored under an unnormalized name as stored', async () => { + const unnormalized = { name: './src//util.js', format: 'TEXT', content: 'util();\n' }; + mockFiles(unnormalized, MAIN_JS); + + await callTool({ operations: [remove('src/main.js', MAIN_JS_HASH)] }); + + expect(getPutFiles()).toStrictEqual([unnormalized]); + }); + + it('sends a path stored twice back once, as the last entry, the one get-actor-version returns', async () => { + const first = { name: 'src/a.js', format: 'TEXT', content: 'first();\n' }; + const last = { name: './src/a.js', format: 'TEXT', content: 'last();\n' }; + mockFiles(first, ACTOR_JSON, last); + + await callTool({ operations: [write('src/b.js', 'b();\n')] }); + + expect(getPutFiles()).toStrictEqual([ + ACTOR_JSON, + last, + { name: 'src/b.js', format: 'TEXT', content: 'b();\n' }, + ]); + }); + }); + + describe('files stored under an unnormalized name or twice', () => { + const UNNORMALIZED = { name: './src//util.js', format: 'TEXT', content: 'util();\n' }; + const UNNORMALIZED_HASH = sha256Prefix(UNNORMALIZED.content); + const FIRST = { name: 'src/a.js', format: 'TEXT', content: 'first();\n' }; + const LAST = { name: './src/a.js', format: 'TEXT', content: 'last();\n' }; + const LAST_HASH = sha256Prefix(LAST.content); + + it('refuses a write without expectedHash to the path an unnormalized name stands for (FILE_EXISTS)', async () => { + mockFiles(UNNORMALIZED, MAIN_JS); + + const text = await callToolExpectingUserError({ operations: [write('src/util.js', 'x')] }); + + expect(text).toBe( + `${failedWith(0, 'write', 'src/util.js', 'FILE_EXISTS')} ` + + `src/util.js exists with hash ${UNNORMALIZED_HASH}; a write with that hash as expectedHash replaces it.`, + ); + }); + + it.each([ + [ + 'replaces', + write('src/util.js', 'x', { expectedHash: UNNORMALIZED_HASH }), + [MAIN_JS, { name: 'src/util.js', format: 'TEXT', content: 'x' }], + ], + [ + 'edits', + edit('src/util.js', { oldText: 'util', newText: 'tool' }), + [MAIN_JS, { name: 'src/util.js', format: 'TEXT', content: 'tool();\n' }], + ], + ['deletes', remove('src/util.js', UNNORMALIZED_HASH), [MAIN_JS]], + ])( + '%s a file stored under an unnormalized name, with no entry left under that name', + async (_, operation, sourceFiles) => { + mockFiles(UNNORMALIZED, MAIN_JS); + + await callTool({ operations: [operation] }); + + expect(versionUpdateMock.mock.calls).toStrictEqual([[{ sourceType: 'SOURCE_FILES', sourceFiles }]]); + }, + ); + + it.each([ + [ + 'replaces', + write('src/a.js', 'x', { expectedHash: LAST_HASH }), + [ACTOR_JSON, { name: 'src/a.js', format: 'TEXT', content: 'x' }], + ], + [ + 'edits', + edit('src/a.js', { oldText: 'last', newText: 'new' }), + [ACTOR_JSON, { name: 'src/a.js', format: 'TEXT', content: 'new();\n' }], + ], + ['deletes', remove('src/a.js', LAST_HASH), [ACTOR_JSON]], + ])( + '%s a file stored twice by its last entry, with neither stored entry left', + async (_, operation, sourceFiles) => { + mockFiles(FIRST, ACTOR_JSON, LAST); + + await callTool({ operations: [operation] }); + + expect(versionUpdateMock.mock.calls).toStrictEqual([[{ sourceType: 'SOURCE_FILES', sourceFiles }]]); + }, + ); + + it('compares expectedHash of a file stored twice with its last entry (HASH_MISMATCH)', async () => { + mockFiles(FIRST, ACTOR_JSON, LAST); + + const text = await callToolExpectingUserError({ + operations: [remove('src/a.js', sha256Prefix(FIRST.content))], + }); + + expect(text).toBe( + `${failedWith(0, 'delete', 'src/a.js', 'HASH_MISMATCH')} ` + + `src/a.js has hash ${LAST_HASH}, not ${sha256Prefix(FIRST.content)}.`, + ); + }); + + it('takes the revision get-actor-version returns for such a version, and sends no PUT when nothing changes', async () => { + mockFiles(UNNORMALIZED, FIRST, ACTOR_JSON, LAST, EMPTY_INIT, NO_CONTENT, FOLDER); + const revision = await readRevision(); + + const unchanged = await callTool({ expectedRevision: revision, operations: [] }); + const rewritten = await callTool({ + expectedRevision: revision, + operations: [write('src/util.js', UNNORMALIZED.content, { expectedHash: UNNORMALIZED_HASH })], + }); + + expect(unchanged.structuredContent).toEqual({ revision, changed: false, changes: [] }); + expect(rewritten.structuredContent).toEqual({ revision, changed: false, changes: [] }); + expect(versionUpdateMock).not.toHaveBeenCalled(); + + const written = await callTool({ expectedRevision: revision, operations: [write('src/b.js', 'b')] }); + + expect(written.structuredContent.changes).toEqual([ + { path: 'src/b.js', action: 'created', hash: sha256Prefix('b') }, + ]); + expect(versionUpdateMock).toHaveBeenCalledTimes(1); + }); + + it('treats an entry with folder false as a file', async () => { + mockFiles({ name: 'src/x.js', folder: false, format: 'TEXT', content: 'x();\n' }); + + const result = await callTool({ operations: [edit('src/x.js', { oldText: 'x()', newText: 'y()' })] }); + + expect(getPutFiles()).toStrictEqual([{ name: 'src/x.js', format: 'TEXT', content: 'y();\n' }]); + expect(result.structuredContent.changes).toEqual([ + { path: 'src/x.js', action: 'updated', hash: sha256Prefix('y();\n') }, + ]); + }); + }); + + describe('PUT body', () => { + it.each([ + ['write', write('src/new.js', 'x')], + ['edit', edit('src/main.js', { oldText: 'const a = 1;', newText: 'const a = 2;' })], + ['delete', remove('src/main.js', MAIN_JS_HASH)], + ])( + 'carries only sourceType and sourceFiles for a %s, whatever else the version holds', + async (_, operation) => { + mockVersionRead({ + applyEnvVarsToBuild: true, + gitRepoUrl: 'https://github.com/john/old-repo.git', + tarballUrl: 'https://example.com/old-tarball.zip', + gitHubGistUrl: 'https://gist.github.com/john/old-gist', + }); + + const result = await callTool({ operations: [operation] }); + + expect(versionUpdateMock).toHaveBeenCalledTimes(1); + expect(versionUpdateMock.mock.calls[0]).toHaveLength(1); + const [body] = versionUpdateMock.mock.calls[0] as [Record]; + expect(Object.keys(body).sort()).toEqual(['sourceFiles', 'sourceType']); + expect(body.sourceType).toBe('SOURCE_FILES'); + expect(JSON.stringify(body)).not.toMatch(/API_KEY|secret-value|latest|old-/); + expect(JSON.stringify(result)).not.toMatch(/API_KEY|secret-value|old-/); + }, + ); + + it('goes to the resolved Actor ID and only the requested version', async () => { + const mainV2 = { name: 'src/main.js', format: 'TEXT', content: 'console.log("v2");\n' }; + actorGetMock.mockResolvedValue({ + id: 'actor-7', + name: 'my-actor', + username: 'john', + versions: [ + mockVersion(), + mockVersion({ versionNumber: '0.2', buildTag: 'beta', sourceFiles: [mainV2] }), + mockVersion({ versionNumber: '0.3', sourceFiles: [ACTOR_JSON] }), + ], + }); + + await callTool({ + versionNumber: '0.2', + autoBuild: true, + operations: [edit('src/main.js', { oldText: 'v2', newText: 'v2 fixed' })], + }); + + expect(actorMock.mock.calls).toEqual([['john/my-actor'], ['actor-7'], ['actor-7']]); + expect(versionMock.mock.calls).toEqual([['0.2']]); + expect(versionUpdateMock.mock.calls).toStrictEqual([ + [ + { + sourceType: 'SOURCE_FILES', + sourceFiles: [{ name: 'src/main.js', format: 'TEXT', content: 'console.log("v2 fixed");\n' }], + }, + ], + ]); + expect(buildMock.mock.calls).toEqual([['0.2', { useCache: true }]]); + }); + + it('matches the version number as a string, so 0.10 is not 0.1', async () => { + const mainV10 = { name: 'src/main.js', format: 'TEXT', content: 'console.log("v10");\n' }; + actorGetMock.mockResolvedValue({ + id: 'actor-1', + name: 'my-actor', + username: 'john', + versions: [mockVersion(), mockVersion({ versionNumber: '0.10', sourceFiles: [mainV10] })], + }); + + await callTool({ + versionNumber: '0.10', + operations: [edit('src/main.js', { oldText: 'v10', newText: 'v11' })], + }); + + expect(versionMock.mock.calls).toEqual([['0.10']]); + expect(versionUpdateMock.mock.calls).toStrictEqual([ + [ + { + sourceType: 'SOURCE_FILES', + sourceFiles: [{ name: 'src/main.js', format: 'TEXT', content: 'console.log("v11");\n' }], + }, + ], + ]); + }); + }); + + describe('write', () => { + it('refuses to replace an existing file without expectedHash (FILE_EXISTS)', async () => { + const text = await callToolExpectingUserError({ operations: [write('src/main.js', MAIN_JS.content)] }); + + expect(text).toBe( + 'Nothing was written: operations[0] (write src/main.js) failed with FILE_EXISTS. ' + + `src/main.js exists with hash ${MAIN_JS_HASH}; a write with that hash as expectedHash replaces it.`, + ); + }); + + it('refuses a stale expectedHash (HASH_MISMATCH)', async () => { + const text = await callToolExpectingUserError({ + operations: [write('src/main.js', 'new\n', { expectedHash: '0000000000000000' })], + }); + + expect(text).toBe( + 'Nothing was written: operations[0] (write src/main.js) failed with HASH_MISMATCH. ' + + `src/main.js has hash ${MAIN_JS_HASH}, not 0000000000000000.`, + ); + }); + + it('refuses expectedHash for a path with no file (FILE_NOT_FOUND)', async () => { + const text = await callToolExpectingUserError({ + operations: [write('src/new.js', 'x', { expectedHash: MAIN_JS_HASH })], + }); + + expect(text).toBe( + 'Nothing was written: operations[0] (write src/new.js) failed with FILE_NOT_FOUND. There is no file at src/new.js.', + ); + }); + + it.each([ + ['lower', MAIN_JS_HASH], + ['upper', MAIN_JS_HASH.toUpperCase()], + ])('replaces a file given its current hash in %s case', async (_, expectedHash) => { + const result = await callTool({ operations: [write('src/main.js', 'new\n', { expectedHash })] }); + + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + LOGO, + { name: 'src/main.js', format: 'TEXT', content: 'new\n' }, + ]); + expect(result.structuredContent.changes).toEqual([ + { path: 'src/main.js', action: 'updated', hash: sha256Prefix('new\n') }, + ]); + }); + + it('needs the hash of the file a path normalizes onto', async () => { + const text = await callToolExpectingUserError({ operations: [write('./src/main.js', 'x')] }); + await callTool({ operations: [write('./src/main.js', 'x', { expectedHash: MAIN_JS_HASH })] }); + + expect(text).toBe( + `${failedWith(0, 'write', './src/main.js', 'FILE_EXISTS')} ` + + `src/main.js exists with hash ${MAIN_JS_HASH}; a write with that hash as expectedHash replaces it.`, + ); + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + LOGO, + { name: 'src/main.js', format: 'TEXT', content: 'x' }, + ]); + }); + + it.each([ + ['assets/icon.png', undefined, 'BASE64'], + ['assets/ICON.PNG', undefined, 'BASE64'], + ['fonts/a.woff2', undefined, 'BASE64'], + ['src/main.ts', undefined, 'TEXT'], + ['assets/icon.svg', undefined, 'TEXT'], + ['Dockerfile', undefined, 'TEXT'], + ['assets/icon.png', 'utf8', 'TEXT'], + ['README.md', 'base64', 'BASE64'], + ])('stores a write to %s with encoding %s as %s, with the content as sent', async (path, encoding, format) => { + const content = 'aGk='; + + await callTool({ operations: [write(path, content, encoding === undefined ? {} : { encoding })] }); + + expect(getPutFiles().at(-1)).toStrictEqual({ name: path, format, content }); + }); + + it.each([ + ['a data URI', 'assets/a.png', 'data:image/png;base64,iVBORw0KGgo=', {}], + ['text sent to a binary extension', 'assets/a.png', 'hello world', {}], + ['text sent with encoding base64', 'README.md', '# Title\n', { encoding: 'base64' }], + ['base64 with stray characters', 'assets/a.png', 'iVBORw0KGgo!!', {}], + ['two base64 strings joined', 'assets/a.png', 'aGk=aGk=', {}], + ])('refuses %s as base64 content, which would be stored corrupted', async (_, path, content, extra) => { + const text = await callToolExpectingUserError({ + autoBuild: true, + operations: [write(path, content, extra)], + }); + + expect(text).toBe( + `operations[0] (write ${path}) has content that is not valid base64; ` + + 'encoding base64 takes binary content as base64, and text takes encoding utf8.', + ); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it.each([ + ['without padding', 'aGk', [104, 105]], + ['split into lines', 'aGVs\nbG8=\n', [104, 101, 108, 108, 111]], + ['in the URL-safe alphabet', '-_8', [251, 255]], + ['empty', '', []], + ])('takes base64 %s, stored as sent and hashed by its bytes', async (_, content, bytes) => { + const result = await callTool({ operations: [write('assets/a.png', content)] }); + + expect(getPutFiles().at(-1)).toStrictEqual({ name: 'assets/a.png', format: 'BASE64', content }); + expect(result.structuredContent.changes).toEqual([ + { path: 'assets/a.png', action: 'created', hash: sha256Prefix(Buffer.from(bytes)) }, + ]); + }); + + it('refuses text with a lone UTF-16 surrogate, which UTF-8 cannot store', async () => { + const text = await callToolExpectingUserError({ operations: [write('src/a.js', 'smile \uD83D')] }); + + expect(text).toBe( + 'operations[0] (write src/a.js) has text with a lone UTF-16 surrogate, which UTF-8 cannot store.', + ); + }); + + it('sends no PUT when the content is the same', async () => { + const result = await callTool({ + operations: [write('assets/logo.png', LOGO.content, { expectedHash: sha256Prefix(LOGO_BYTES) })], + }); + + expect(versionUpdateMock).not.toHaveBeenCalled(); + expect(result.structuredContent).toEqual({ + revision: await readRevision(), + changed: false, + changes: [], + }); + expect(result.content[1].text).toBe( + 'Nothing changed in version 0.1 of john/my-actor, so nothing was written.\nNo build was started.', + ); + }); + + it('stores content for a binary extension as BASE64 by default, and text sent as base64 as BASE64', async () => { + const icon = Buffer.from([1, 2, 3]).toString('base64'); + const readme = Buffer.from('hello\n').toString('base64'); + const result = await callTool({ + operations: [write('assets/icon.png', icon), write('README.md', readme, { encoding: 'base64' })], + }); + + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + LOGO, + MAIN_JS, + { name: 'assets/icon.png', format: 'BASE64', content: icon }, + { name: 'README.md', format: 'BASE64', content: readme }, + ]); + expect(result.structuredContent.changes).toContainEqual({ + path: 'assets/icon.png', + action: 'created', + hash: sha256Prefix(Buffer.from([1, 2, 3])), + }); + }); + + it('stores a write with no encoding over a UTF-8 file stored as BASE64 as TEXT, with the content as sent', async () => { + const readme = { name: 'README.md', format: 'BASE64', content: Buffer.from('# Hi\n').toString('base64') }; + mockFiles(readme, MAIN_JS); + + const result = await callTool({ + operations: [write('README.md', '# Bye\n', { expectedHash: sha256Prefix('# Hi\n') })], + }); + + expect(getPutFiles()).toStrictEqual([{ name: 'README.md', format: 'TEXT', content: '# Bye\n' }, MAIN_JS]); + expect(result.structuredContent.changes).toEqual([ + { path: 'README.md', action: 'updated', hash: sha256Prefix('# Bye\n') }, + ]); + }); + + it.each([ + ['no encoding', {}], + ['encoding utf8', { encoding: 'utf8' }], + ])('stores text sent with %s byte for byte', async (_, extra) => { + const content = '\uFEFFline one \r\nn\u00e1zev = "🙂"\n\tend\r'; + + const result = await callTool({ operations: [write('src/win.js', content, extra)] }); + + expect(getPutFiles().at(-1)).toStrictEqual({ name: 'src/win.js', format: 'TEXT', content }); + expect(result.structuredContent.changes).toEqual([ + { path: 'src/win.js', action: 'created', hash: sha256Prefix(Buffer.from(content)) }, + ]); + }); + + it('stores a non-ASCII path as sent, with the revision get-actor-version reads, and takes its NFD form as another file', async () => { + const nfcPath = 'src/název.js'; + const nfdPath = 'src/název.js'; + mockFiles(MAIN_JS); + + const result = await callTool({ operations: [write(nfcPath, 'x')] }); + const putFiles = getPutFiles(); + mockFiles(...putFiles); + + expect(putFiles).toStrictEqual([MAIN_JS, { name: nfcPath, format: 'TEXT', content: 'x' }]); + expect(result.structuredContent.revision).toBe(await readRevision()); + + versionUpdateMock.mockClear(); + const nfd = await callTool({ operations: [write(nfdPath, 'y')] }); + + expect(getPutFiles()).toStrictEqual([...putFiles, { name: nfdPath, format: 'TEXT', content: 'y' }]); + expect(nfd.structuredContent.changes).toEqual([ + { path: nfdPath, action: 'created', hash: sha256Prefix('y') }, + ]); + }); + + it('writes the first file of a version with no files', async () => { + mockFiles(); + + const result = await callTool({ operations: [write('a.js', 'a')] }); + + expect(versionUpdateMock.mock.calls).toStrictEqual([ + [{ sourceType: 'SOURCE_FILES', sourceFiles: [{ name: 'a.js', format: 'TEXT', content: 'a' }] }], + ]); + expect(result.structuredContent.changes).toEqual([ + { path: 'a.js', action: 'created', hash: sha256Prefix('a') }, + ]); + }); + + it('saves an empty file and warns that the build skips it', async () => { + const result = await callTool({ operations: [write('src/__init__.py', '')] }); + + expectSchemaConformingStructuredContent(result, updateActorVersionToolOutputSchema); + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + LOGO, + MAIN_JS, + { name: 'src/__init__.py', format: 'TEXT', content: '' }, + ]); + expect(result.structuredContent.warnings).toEqual([ + 'These files are empty, and the build skips empty files, so they will not exist in the build: src/__init__.py.', + ]); + }); + + it('saves every empty file, and warns about the ones the call leaves empty, not the ones it leaves alone', async () => { + mockFiles(ACTOR_JSON, MAIN_JS, EMPTY_INIT, NO_CONTENT); + + const result = await callTool({ + operations: [ + edit('src/main.js', { oldText: MAIN_JS.content, newText: '' }), + write('assets/blank.png', ''), + ], + }); + + expect(getPutFiles()).toStrictEqual([ + ACTOR_JSON, + EMPTY_INIT, + NO_CONTENT, + { name: 'src/main.js', format: 'TEXT', content: '' }, + { name: 'assets/blank.png', format: 'BASE64', content: '' }, + ]); + expect(result.structuredContent.warnings).toEqual([ + 'These files are empty, and the build skips empty files, so they will not exist in the build: assets/blank.png, src/main.js.', + ]); + }); + + it('gives no warning when no file is left empty', async () => { + const result = await callTool({ operations: [write('src/b.js', 'b')] }); + + expect(result.structuredContent).not.toHaveProperty('warnings'); + }); + }); + + describe('edit', () => { + it('applies edits in order, each to the text the previous one left', async () => { + const result = await callTool({ + operations: [ + edit( + 'src/main.js', + { oldText: 'const a = 1;', newText: 'const a = 2;' }, + { oldText: 'a = 2', newText: 'a = 3' }, + ), + ], + }); + + const content = 'const a = 3;\nconsole.log(a);\nexport {};\n'; + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + LOGO, + { name: 'src/main.js', format: 'TEXT', content }, + ]); + expect(result.structuredContent.changes).toEqual([ + { path: 'src/main.js', action: 'updated', hash: sha256Prefix(content) }, + ]); + }); + + it('keeps every edit of an operation that changes different parts of the file', async () => { + const result = await callTool({ + operations: [ + edit( + 'src/main.js', + { oldText: 'const a = 1;', newText: 'const a = 22;' }, + { oldText: 'export {};', newText: 'export { a };' }, + ), + ], + }); + + const content = 'const a = 22;\nconsole.log(a);\nexport { a };\n'; + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + LOGO, + { name: 'src/main.js', format: 'TEXT', content }, + ]); + expect(result.structuredContent.changes).toEqual([ + { path: 'src/main.js', action: 'updated', hash: sha256Prefix(content) }, + ]); + }); + + it('inserts newText as given, with no replacement patterns', async () => { + await callTool({ + operations: [edit('src/main.js', { oldText: 'const a = 1;', newText: "$&|$1|$$|$'|$`|$" })], + }); + + expect(getPutFiles()).toContainEqual({ + name: 'src/main.js', + format: 'TEXT', + content: "$&|$1|$$|$'|$`|$\nconsole.log(a);\nexport {};\n", + }); + }); + + it.each([ + ['at the start of the file', 'const a', 'let a', 'let a = 1;\nconsole.log(a);\nexport {};\n'], + [ + 'at the end of the file', + 'export {};\n', + 'export { a };\n', + 'const a = 1;\nconsole.log(a);\nexport { a };\n', + ], + ['equal to the whole file', MAIN_JS.content, 'x\n', 'x\n'], + ])('replaces an oldText %s', async (_, oldText, newText, content) => { + await callTool({ operations: [edit('src/main.js', { oldText, newText })] }); + + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + LOGO, + { name: 'src/main.js', format: 'TEXT', content }, + ]); + }); + + it('reports the edit of an operation that no longer matches after the edits before it', async () => { + const text = await callToolExpectingUserError({ + operations: [ + edit( + 'src/main.js', + { oldText: 'const a = 1;', newText: 'const a = 2;' }, + { oldText: 'const a = 1;', newText: 'const a = 3;' }, + ), + ], + }); + + expect(text).toBe( + `${failedWith(0, 'edit', 'src/main.js', 'NO_MATCH')} oldText of edits[1] is not in the file.`, + ); + }); + + it.each([ + ['});\n});\n});\n', '});\n});'], + ['aaa\n', 'aa'], + ])('reports an oldText whose matches overlap in %j (MULTIPLE_MATCHES)', async (content, oldText) => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content }); + + const text = await callToolExpectingUserError({ + operations: [edit('src/main.js', { oldText, newText: '' })], + }); + + expect(text).toBe( + `${failedWith(0, 'edit', 'src/main.js', 'MULTIPLE_MATCHES')} ` + + 'oldText of edits[0] matches more than once; surrounding lines in oldText can narrow it to one match.', + ); + }); + + it('reports an oldText longer than 250 characters whose matches overlap (MULTIPLE_MATCHES)', async () => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content: 'x'.repeat(301) }); + + const text = await callToolExpectingUserError({ + operations: [edit('src/main.js', { oldText: 'x'.repeat(300), newText: '' })], + }); + + expect(text).toBe( + `${failedWith(0, 'edit', 'src/main.js', 'MULTIPLE_MATCHES')} ` + + 'oldText of edits[0] matches more than once; surrounding lines in oldText can narrow it to one match.', + ); + }); + + it('applies an oldText longer than 250 characters that matches once, after partial matches', async () => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content: `${'ab'.repeat(500)}c` }); + + await callTool({ operations: [edit('src/main.js', { oldText: `${'ab'.repeat(200)}c`, newText: 'X' })] }); + + expect(getPutFiles()).toStrictEqual([ + { name: 'src/main.js', format: 'TEXT', content: `${'ab'.repeat(300)}X` }, + ]); + }); + + it('reports an oldText that the edits before it made match twice (MULTIPLE_MATCHES)', async () => { + const text = await callToolExpectingUserError({ + operations: [ + edit( + 'src/main.js', + { oldText: 'export {};', newText: 'console.log(a);\nexport {};' }, + { oldText: 'console.log(a);', newText: 'console.log(b);' }, + ), + ], + }); + + expect(text).toBe( + `${failedWith(0, 'edit', 'src/main.js', 'MULTIPLE_MATCHES')} ` + + 'oldText of edits[1] matches more than once; surrounding lines in oldText can narrow it to one match.', + ); + }); + + it('applies an oldText that matched twice before the edits before it removed one match', async () => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content: 'a();\nx();\nb();\nx();\n' }); + + await callTool({ + operations: [ + edit( + 'src/main.js', + { oldText: 'b();\nx();\n', newText: 'b();\n' }, + { oldText: 'x();', newText: 'y();' }, + ), + ], + }); + + expect(getPutFiles()).toStrictEqual([ + { name: 'src/main.js', format: 'TEXT', content: 'a();\ny();\nb();\n' }, + ]); + }); + + it('edits a file with a binary extension stored as TEXT', async () => { + mockFiles({ name: 'data/notes.dat', format: 'TEXT', content: 'a=1\n' }); + + await callTool({ operations: [edit('data/notes.dat', { oldText: 'a=1', newText: 'a=2' })] }); + + expect(getPutFiles()).toStrictEqual([{ name: 'data/notes.dat', format: 'TEXT', content: 'a=2\n' }]); + }); + + it('refuses to edit a BASE64 file with a binary extension even when its bytes are UTF-8 (NOT_TEXT)', async () => { + mockFiles({ name: 'assets/x.png', format: 'BASE64', content: Buffer.from('hello').toString('base64') }); + + const text = await callToolExpectingUserError({ + operations: [edit('assets/x.png', { oldText: 'hello', newText: 'bye' })], + }); + + expect(text).toBe( + `${failedWith(0, 'edit', 'assets/x.png', 'NOT_TEXT')} assets/x.png is not UTF-8 text; only a write can replace it.`, + ); + }); + + it('edits a file at a path that normalizes onto it', async () => { + await callTool({ + operations: [edit('./src//main.js', { oldText: 'const a = 1;', newText: 'const a = 2;' })], + }); + + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + LOGO, + { name: 'src/main.js', format: 'TEXT', content: 'const a = 2;\nconsole.log(a);\nexport {};\n' }, + ]); + }); + + it('refuses an edit given a stale expectedHash (HASH_MISMATCH)', async () => { + const text = await callToolExpectingUserError({ + autoBuild: true, + operations: [ + { + ...edit('src/main.js', { oldText: 'const a = 1;', newText: 'const a = 2;' }), + expectedHash: '0000000000000000', + }, + ], + }); + + expect(text).toBe( + `${failedWith(0, 'edit', 'src/main.js', 'HASH_MISMATCH')} src/main.js has hash ${MAIN_JS_HASH}, not 0000000000000000.`, + ); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it.each([ + ['lower', MAIN_JS_HASH], + ['upper', MAIN_JS_HASH.toUpperCase()], + ])('edits a file given its current hash in %s case', async (_, expectedHash) => { + await callTool({ + operations: [ + { ...edit('src/main.js', { oldText: 'const a = 1;', newText: 'const a = 2;' }), expectedHash }, + ], + }); + + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + LOGO, + { name: 'src/main.js', format: 'TEXT', content: 'const a = 2;\nconsole.log(a);\nexport {};\n' }, + ]); + }); + + it.each([ + ['an oldText that is half of an emoji', { oldText: '\uD83D', newText: 'x' }], + ['a newText that is half of an emoji', { oldText: 'smile', newText: '\uDE42' }], + ])('refuses %s, which would store U+FFFD', async (_, textEdit) => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content: 'smile 🙂\n' }); + + const text = await callToolExpectingUserError({ + operations: [edit('src/main.js', { oldText: 'smile', newText: 'smile' }, textEdit)], + }); + + expect(text).toBe( + 'operations[0] (edit src/main.js) has a lone UTF-16 surrogate in edits[1], which UTF-8 cannot store.', + ); + }); + + it('reports an oldText that is not in the file (NO_MATCH)', async () => { + const text = await callToolExpectingUserError({ + operations: [edit('src/main.js', { oldText: 'zzz', newText: 'y' })], + }); + + expect(text).toBe( + 'Nothing was written: operations[0] (edit src/main.js) failed with NO_MATCH. oldText of edits[0] is not in the file.', + ); + }); + + it('reports an oldText that matches more than once (MULTIPLE_MATCHES)', async () => { + const text = await callToolExpectingUserError({ + operations: [edit('src/main.js', { oldText: 'a', newText: 'b' })], + }); + + expect(text).toBe( + 'Nothing was written: operations[0] (edit src/main.js) failed with MULTIPLE_MATCHES. ' + + 'oldText of edits[0] matches more than once; surrounding lines in oldText can narrow it to one match.', + ); + }); + + it('takes LF in oldText and newText as CRLF in a file with only CRLF line breaks', async () => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content: 'one\r\ntwo\r\nthree\r\n' }); + + await callTool({ operations: [edit('src/main.js', { oldText: 'one\ntwo\n', newText: '1\n2\n' })] }); + + expect(getPutFiles()).toEqual([{ name: 'src/main.js', format: 'TEXT', content: '1\r\n2\r\nthree\r\n' }]); + }); + + it('converts LF in newText to CRLF, and keeps the CRLF already in it', async () => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content: 'one\r\ntwo\r\nthree\r\n' }); + + await callTool({ operations: [edit('src/main.js', { oldText: 'two\nthree', newText: 'a\r\nb\nc' })] }); + + expect(getPutFiles()).toEqual([{ name: 'src/main.js', format: 'TEXT', content: 'one\r\na\r\nb\r\nc\r\n' }]); + }); + + it('requires an oldText taken as CRLF to match once', async () => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content: 'a\r\nb\r\na\r\nb\r\n' }); + + const text = await callToolExpectingUserError({ + operations: [edit('src/main.js', { oldText: 'a\nb', newText: 'x' })], + }); + + expect(text).toContain(failedWith(0, 'edit', 'src/main.js', 'MULTIPLE_MATCHES')); + }); + + it('takes an oldText that starts with LF as CRLF, so the edit leaves no lone CR', async () => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content: 'one\r\ntwo\r\n' }); + + await callTool({ operations: [edit('src/main.js', { oldText: '\ntwo', newText: 'X' })] }); + + expect(getPutFiles()).toEqual([{ name: 'src/main.js', format: 'TEXT', content: 'oneX\r\n' }]); + }); + + it('converts LF in newText to CRLF when oldText has no line break, so line endings stay CRLF', async () => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content: 'one\r\ntwo\r\n' }); + + await callTool({ operations: [edit('src/main.js', { oldText: 'two', newText: 'two\nmore' })] }); + + expect(getPutFiles()).toEqual([{ name: 'src/main.js', format: 'TEXT', content: 'one\r\ntwo\r\nmore\r\n' }]); + }); + + it('takes the LF in an oldText that also has CRLF as CRLF', async () => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content: 'one\r\ntwo\r\n' }); + + await callTool({ operations: [edit('src/main.js', { oldText: 'one\r\ntwo\n', newText: 'x' })] }); + + expect(getPutFiles()).toEqual([{ name: 'src/main.js', format: 'TEXT', content: 'x' }]); + }); + + it('keeps taking LF as CRLF after an earlier edit of the operation removed every line break', async () => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content: 'a\r\nb\r\n' }); + + await callTool({ + operations: [ + edit('src/main.js', { oldText: 'a\nb\n', newText: 'ab' }, { oldText: 'ab', newText: 'a\nc\n' }), + ], + }); + + expect(getPutFiles()).toEqual([{ name: 'src/main.js', format: 'TEXT', content: 'a\r\nc\r\n' }]); + }); + + it.each([ + ['no line break', 'one two', 'one two\nthree'], + ['mixed line endings', 'one\r\ntwo\n', 'one\r\ntwo\nthree\n'], + ])('keeps LF in newText as LF in a file with %s', async (_, content, edited) => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content }); + + await callTool({ operations: [edit('src/main.js', { oldText: 'two', newText: 'two\nthree' })] }); + + expect(getPutFiles()).toEqual([{ name: 'src/main.js', format: 'TEXT', content: edited }]); + }); + + it('does not convert line endings in a file with mixed line endings', async () => { + mockFiles({ name: 'src/main.js', format: 'TEXT', content: 'one\r\ntwo\nthree\r\n' }); + + const text = await callToolExpectingUserError({ + operations: [edit('src/main.js', { oldText: 'one\ntwo', newText: 'x' })], + }); + + expect(text).toBe( + `${failedWith(0, 'edit', 'src/main.js', 'NO_MATCH')} oldText of edits[0] is not in the file.`, + ); + }); + + it('keeps a UTF-8 file stored as BASE64 in BASE64', async () => { + mockFiles({ name: 'src/data.txt', format: 'BASE64', content: Buffer.from('hello\n').toString('base64') }); + + await callTool({ operations: [edit('src/data.txt', { oldText: 'hello', newText: 'bye' })] }); + + expect(getPutFiles()).toEqual([ + { name: 'src/data.txt', format: 'BASE64', content: Buffer.from('bye\n').toString('base64') }, + ]); + }); + + it('stores the exact UTF-8 bytes of an edited BASE64 file, byte order mark included', async () => { + const before = Buffer.from('název = "č"\n'); + mockFiles({ name: 'src/data.txt', format: 'BASE64', content: before.toString('base64') }); + + const result = await callTool({ operations: [edit('src/data.txt', { oldText: 'č', newText: '🙂' })] }); + + const after = Buffer.from([0xef, 0xbb, 0xbf, ...Buffer.from('název = "🙂"\n')]); + expect(getPutFiles()).toEqual([ + { name: 'src/data.txt', format: 'BASE64', content: after.toString('base64') }, + ]); + expect(result.structuredContent.changes).toEqual([ + { path: 'src/data.txt', action: 'updated', hash: sha256Prefix(after) }, + ]); + }); + + it('refuses to edit a file with a text extension whose bytes are not UTF-8 (NOT_TEXT)', async () => { + mockFiles({ + name: 'src/data.txt', + format: 'BASE64', + content: Buffer.from([0xff, 0xfe, 0x41]).toString('base64'), + }); + + const text = await callToolExpectingUserError({ + operations: [edit('src/data.txt', { oldText: 'A', newText: 'B' })], + }); + + expect(text).toBe( + `${failedWith(0, 'edit', 'src/data.txt', 'NOT_TEXT')} src/data.txt is not UTF-8 text; only a write can replace it.`, + ); + }); + + it('refuses to edit a binary file (NOT_TEXT)', async () => { + const text = await callToolExpectingUserError({ + operations: [edit('assets/logo.png', { oldText: 'PNG', newText: 'x' })], + }); + + expect(text).toBe( + 'Nothing was written: operations[0] (edit assets/logo.png) failed with NOT_TEXT. ' + + 'assets/logo.png is not UTF-8 text; only a write can replace it.', + ); + }); + + it('refuses to edit a missing file (FILE_NOT_FOUND)', async () => { + const text = await callToolExpectingUserError({ + operations: [edit('src/none.js', { oldText: 'a', newText: 'b' })], + }); + + expect(text).toContain('(edit src/none.js) failed with FILE_NOT_FOUND. There is no file at src/none.js.'); + }); + }); + + describe('delete', () => { + it('deletes a file given its hash', async () => { + const result = await callTool({ + operations: [{ type: 'delete', path: 'src/main.js', expectedHash: MAIN_JS_HASH }], + }); + + expect(getPutFiles()).toStrictEqual([FOLDER, ACTOR_JSON, LOGO]); + // No other key: a deleted file is not named as an empty one. + expect(result.structuredContent).toStrictEqual({ + revision: expect.stringMatching(/^[0-9a-f]{16}$/), + changed: true, + changes: [{ path: 'src/main.js', action: 'deleted' }], + }); + expectSchemaConformingStructuredContent(result, updateActorVersionToolOutputSchema); + }); + + it('names no deleted empty file as empty', async () => { + mockFiles(MAIN_JS, EMPTY_INIT, NO_CONTENT); + + const result = await callTool({ + operations: [remove('src/__init__.py', sha256Prefix('')), remove('src/blank.js', sha256Prefix(''))], + }); + + expect(getPutFiles()).toStrictEqual([MAIN_JS]); + expect(result.structuredContent).toStrictEqual({ + revision: buildFilesRevision([{ path: 'src/main.js', hash: MAIN_JS_HASH }]), + changed: true, + changes: [ + { path: 'src/__init__.py', action: 'deleted' }, + { path: 'src/blank.js', action: 'deleted' }, + ], + }); + }); + + it('deletes the last file, leaving no files and the folder entries', async () => { + mockFiles(MAIN_JS, FOLDER); + + const result = await callTool({ operations: [remove('src/main.js', MAIN_JS_HASH)] }); + + expect(versionUpdateMock.mock.calls).toStrictEqual([ + [{ sourceType: 'SOURCE_FILES', sourceFiles: [FOLDER] }], + ]); + expect(result.structuredContent.changes).toEqual([{ path: 'src/main.js', action: 'deleted' }]); + + mockFiles(MAIN_JS); + versionUpdateMock.mockClear(); + await callTool({ operations: [remove('src/main.js', MAIN_JS_HASH)] }); + + expect(versionUpdateMock.mock.calls).toStrictEqual([[{ sourceType: 'SOURCE_FILES', sourceFiles: [] }]]); + }); + + it('refuses a stale hash (HASH_MISMATCH) and a missing file (FILE_NOT_FOUND)', async () => { + const stale = await callToolExpectingUserError({ + operations: [{ type: 'delete', path: 'src/main.js', expectedHash: 'ffffffffffffffff' }], + }); + const missing = await callToolExpectingUserError({ + operations: [{ type: 'delete', path: 'src/none.js', expectedHash: MAIN_JS_HASH }], + }); + + expect(stale).toBe( + `${failedWith(0, 'delete', 'src/main.js', 'HASH_MISMATCH')} src/main.js has hash ${MAIN_JS_HASH}, not ffffffffffffffff.`, + ); + expect(missing).toBe( + `${failedWith(0, 'delete', 'src/none.js', 'FILE_NOT_FOUND')} There is no file at src/none.js.`, + ); + }); + + it('deletes a file given its hash in upper case', async () => { + await callTool({ operations: [remove('src/main.js', MAIN_JS_HASH.toUpperCase())] }); + + expect(getPutFiles()).toStrictEqual([FOLDER, ACTOR_JSON, LOGO]); + }); + + it('renames a file with a delete and a write', async () => { + const result = await callTool({ + operations: [ + { type: 'delete', path: 'src/main.js', expectedHash: MAIN_JS_HASH }, + write('src/index.js', MAIN_JS.content), + ], + }); + + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + LOGO, + { name: 'src/index.js', format: 'TEXT', content: MAIN_JS.content }, + ]); + expect(result.structuredContent.changes).toEqual([ + { path: 'src/index.js', action: 'created', hash: MAIN_JS_HASH }, + { path: 'src/main.js', action: 'deleted' }, + ]); + }); + }); + + describe('operation order', () => { + const editedMainJs = 'const a = 2;\nconsole.log(a);\nexport {};\n'; + const editMainJs = edit('src/main.js', { oldText: 'const a = 1;', newText: 'const a = 2;' }); + + it('edits a file an earlier write created or replaced, with no hash for the edit', async () => { + const result = await callTool({ + operations: [ + write('src/new.js', 'hello\n'), + edit('src/new.js', { oldText: 'hello', newText: 'bye' }), + write('src/main.js', 'x = 1;\n', { expectedHash: MAIN_JS_HASH }), + edit('src/main.js', { oldText: '1', newText: '2' }), + ], + }); + + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + LOGO, + { name: 'src/main.js', format: 'TEXT', content: 'x = 2;\n' }, + { name: 'src/new.js', format: 'TEXT', content: 'bye\n' }, + ]); + expect(result.structuredContent.changes).toEqual([ + { path: 'src/main.js', action: 'updated', hash: sha256Prefix('x = 2;\n') }, + { path: 'src/new.js', action: 'created', hash: sha256Prefix('bye\n') }, + ]); + }); + + it('applies an edit operation to the text an earlier edit operation on the file left', async () => { + const stale = await callToolExpectingUserError({ + operations: [editMainJs, edit('src/main.js', { oldText: 'const a = 1;', newText: 'const a = 3;' })], + }); + const result = await callTool({ + operations: [editMainJs, edit('src/main.js', { oldText: 'const a = 2;', newText: 'const a = 3;' })], + }); + + const content = 'const a = 3;\nconsole.log(a);\nexport {};\n'; + expect(getPutFiles()).toContainEqual({ name: 'src/main.js', format: 'TEXT', content }); + expect(result.structuredContent.changes).toEqual([ + { path: 'src/main.js', action: 'updated', hash: sha256Prefix(content) }, + ]); + expect(stale).toBe( + `${failedWith(1, 'edit', 'src/main.js', 'NO_MATCH')} oldText of edits[0] is not in the file.`, + ); + }); + + it('compares the hash of a delete after an edit with the edited file', async () => { + const stale = await callToolExpectingUserError({ + operations: [editMainJs, remove('src/main.js', MAIN_JS_HASH)], + }); + const result = await callTool({ + operations: [editMainJs, remove('src/main.js', sha256Prefix(editedMainJs))], + }); + + expect(stale).toBe( + `${failedWith(1, 'delete', 'src/main.js', 'HASH_MISMATCH')} ` + + `src/main.js has hash ${sha256Prefix(editedMainJs)}, not ${MAIN_JS_HASH}.`, + ); + expect(getPutFiles()).toStrictEqual([FOLDER, ACTOR_JSON, LOGO]); + expect(result.structuredContent.changes).toEqual([{ path: 'src/main.js', action: 'deleted' }]); + }); + + it('takes a write after a delete of the same path as a new file, which needs no hash', async () => { + const withHash = await callToolExpectingUserError({ + operations: [ + remove('src/main.js', MAIN_JS_HASH), + write('src/main.js', 'new\n', { expectedHash: MAIN_JS_HASH }), + ], + }); + const result = await callTool({ + operations: [remove('src/main.js', MAIN_JS_HASH), write('src/main.js', 'new\n')], + }); + + expect(withHash).toBe( + `${failedWith(1, 'write', 'src/main.js', 'FILE_NOT_FOUND')} There is no file at src/main.js.`, + ); + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + LOGO, + { name: 'src/main.js', format: 'TEXT', content: 'new\n' }, + ]); + expect(result.structuredContent.changes).toEqual([ + { path: 'src/main.js', action: 'updated', hash: sha256Prefix('new\n') }, + ]); + }); + + it('compares the hash of a second write with the content of the first', async () => { + const stale = await callToolExpectingUserError({ + operations: [ + write('src/main.js', 'v1\n', { expectedHash: MAIN_JS_HASH }), + write('src/main.js', 'v2\n', { expectedHash: MAIN_JS_HASH }), + ], + }); + const exists = await callToolExpectingUserError({ + operations: [write('src/new.js', 'v1\n'), write('src/new.js', 'v2\n')], + }); + await callTool({ + operations: [ + write('src/main.js', 'v1\n', { expectedHash: MAIN_JS_HASH }), + write('src/main.js', 'v2\n', { expectedHash: sha256Prefix('v1\n') }), + ], + }); + + expect(stale).toBe( + `${failedWith(1, 'write', 'src/main.js', 'HASH_MISMATCH')} ` + + `src/main.js has hash ${sha256Prefix('v1\n')}, not ${MAIN_JS_HASH}.`, + ); + expect(exists).toBe( + `${failedWith(1, 'write', 'src/new.js', 'FILE_EXISTS')} ` + + `src/new.js exists with hash ${sha256Prefix('v1\n')}; a write with that hash as expectedHash replaces it.`, + ); + expect(getPutFiles()).toContainEqual({ name: 'src/main.js', format: 'TEXT', content: 'v2\n' }); + }); + }); + + describe('input', () => { + it.each(['/src/main.js', '..', '../x.js', 'src/../../x.js', 'a/../..'])( + 'refuses a write to %s, outside the Actor root, which the build refuses', + async (path) => { + const text = await callToolExpectingUserError({ autoBuild: true, operations: [write(path, 'x')] }); + + expect(text).toBe( + `operations[0] (write ${path}) has a path outside the Actor root; paths are relative to it, such as src/main.js.`, + ); + expect(buildMock).not.toHaveBeenCalled(); + }, + ); + + it.each(['src\\main.js', '.\\Dockerfile'])( + 'refuses a write to %s, whose backslash the build takes as part of the file name', + async (path) => { + const text = await callToolExpectingUserError({ autoBuild: true, operations: [write(path, 'x')] }); + + expect(text).toBe( + `operations[0] (write ${path}) has a backslash in its path; paths use / as the separator.`, + ); + expect(buildMock).not.toHaveBeenCalled(); + }, + ); + + it.each([ + ['U+0000', 'src/a\0.js'], + ['U+000A', 'src/\nmain.js'], + ['U+0009', 'src/a\tb.js'], + ['U+001F', 'src/a\u001f.js'], + ['U+007F', 'src/a\u007f.js'], + ])( + 'refuses a path with the control character %s in any operation, before any request', + async (character, path) => { + const text = await callToolExpectingUserError({ + autoBuild: true, + operations: [ + write('src/ok.js', 'x'), + edit(path, { oldText: 'a', newText: 'b' }), + remove(path, MAIN_JS_HASH), + write(path, 'x'), + ], + }); + + expect(text).toBe( + `operations[1] has the control character ${character} in its path; ` + + 'a path cannot hold characters U+0000 to U+001F or U+007F.', + ); + expect(actorGetMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }, + ); + + it.each([ + ['write', write('src/a\0.js', 'x')], + ['edit', edit('src/a\0.js', { oldText: 'a', newText: 'b' })], + ['delete', remove('src/a\0.js', MAIN_JS_HASH)], + ])('refuses a NUL in the path of a single %s operation, before any request', async (_, operation) => { + const text = await callToolExpectingUserError({ autoBuild: true, operations: [operation] }); + + expect(text).toBe( + 'operations[0] has the control character U+0000 in its path; ' + + 'a path cannot hold characters U+0000 to U+001F or U+007F.', + ); + expect(actorGetMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('refuses a path with a lone UTF-16 surrogate in any operation, before any request', async () => { + const text = await callToolExpectingUserError({ + autoBuild: true, + operations: [ + write('src/ok.js', 'x'), + edit('src/a\uD83D.js', { oldText: 'a', newText: 'b' }), + write('src/b\uDE42.js', 'x'), + ], + }); + + expect(text).toBe('operations[1] has a lone UTF-16 surrogate in its path, which UTF-8 cannot store.'); + expect(actorGetMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('refuses the delete, delete, and write whose file sets once gave one revision, before any request', async () => { + // Joined as path, NUL, hash, and a line break per file, the files before and after read the same, so the + // call reported changes and sent no PUT. + mockFiles({ name: 'a', format: 'TEXT', content: 'A' }, { name: 'b', format: 'TEXT', content: 'B' }); + const path = `a\0${sha256Prefix('A')}\nb`; + + const text = await callToolExpectingUserError({ + operations: [remove('a', sha256Prefix('A')), remove('b', sha256Prefix('B')), write(path, 'B')], + }); + + expect(text).toBe( + 'operations[2] has the control character U+0000 in its path; ' + + 'a path cannot hold characters U+0000 to U+001F or U+007F.', + ); + expect(actorGetMock).not.toHaveBeenCalled(); + }); + + it.each(['.', './', 'src/', 'src//', 'src/..'])('refuses a write to %s, which names a folder', async (path) => { + const text = await callToolExpectingUserError({ autoBuild: true, operations: [write(path, 'x')] }); + + expect(text).toBe(`operations[0] (write ${path}) has a path that names a folder, not a file.`); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it.each([ + ['storage', 'storage'], + ['src', 'src/main.js'], + ['src/main.js/inner.js', 'src/main.js'], + ['assets/logo.png/x', 'assets/logo.png'], + ])( + 'refuses a write to %s, which collides with %s, since a path cannot be both a file and a folder', + async (path, collision) => { + const text = await callToolExpectingUserError({ autoBuild: true, operations: [write(path, 'x')] }); + + expect(text).toBe( + `operations[0] (write ${path}) collides with ${collision}; one path cannot be both a file and a folder.`, + ); + expect(buildMock).not.toHaveBeenCalled(); + }, + ); + + it('refuses a write under a file an earlier write created', async () => { + const text = await callToolExpectingUserError({ + operations: [write('lib', 'x'), write('lib/util.js', 'y')], + }); + + expect(text).toBe( + 'operations[1] (write lib/util.js) collides with lib; one path cannot be both a file and a folder.', + ); + }); + + it('refuses a write to a folder stored with a trailing slash or an unnormalized name, and to a parent of a nested folder', async () => { + mockFiles( + { name: 'cache/', folder: true }, + { name: 'data/raw', folder: true }, + { name: './storage', folder: true }, + MAIN_JS, + ); + + const cache = await callToolExpectingUserError({ operations: [write('cache', 'x')] }); + const data = await callToolExpectingUserError({ operations: [write('data', 'x')] }); + const storage = await callToolExpectingUserError({ operations: [write('storage', 'x')] }); + + expect(cache).toBe( + 'operations[0] (write cache) collides with cache/; one path cannot be both a file and a folder.', + ); + expect(data).toBe( + 'operations[0] (write data) collides with data/raw; one path cannot be both a file and a folder.', + ); + expect(storage).toBe( + 'operations[0] (write storage) collides with storage; one path cannot be both a file and a folder.', + ); + }); + + it('writes a file into a folder entry, and where a file deleted in the same call held a folder', async () => { + await callTool({ + operations: [ + write('storage/input.json', '{}'), + remove('src/main.js', MAIN_JS_HASH), + write('src', 'x'), + remove('assets/logo.png', sha256Prefix(LOGO_BYTES)), + write('assets/logo.png/readme.txt', 'y'), + ], + }); + + expect(getPutFiles()).toStrictEqual([ + FOLDER, + ACTOR_JSON, + { name: 'storage/input.json', format: 'TEXT', content: '{}' }, + { name: 'src', format: 'TEXT', content: 'x' }, + { name: 'assets/logo.png/readme.txt', format: 'TEXT', content: 'y' }, + ]); + }); + + it('refuses to delete or edit a folder (FILE_NOT_FOUND)', async () => { + const deleted = await callToolExpectingUserError({ operations: [remove('storage', MAIN_JS_HASH)] }); + const edited = await callToolExpectingUserError({ + operations: [edit('storage', { oldText: 'a', newText: 'b' })], + }); + + expect(deleted).toBe( + `${failedWith(0, 'delete', 'storage', 'FILE_NOT_FOUND')} There is no file at storage.`, + ); + expect(edited).toBe(`${failedWith(0, 'edit', 'storage', 'FILE_NOT_FOUND')} There is no file at storage.`); + }); + + it.each([ + ['an edit', edit('/src/main.js', { oldText: 'const a = 1;', newText: 'const a = 2;' })], + ['a delete', remove('/src/main.js', MAIN_JS_HASH)], + ])('takes the path of %s with a leading slash as written, so it finds no file', async (_, operation) => { + const text = await callToolExpectingUserError({ operations: [operation] }); + + expect(text).toBe( + `${failedWith(0, operation.type, '/src/main.js', 'FILE_NOT_FOUND')} There is no file at /src/main.js.`, + ); + }); + + it('deletes a stored file outside the Actor root, so such a file can be removed', async () => { + const outside = { name: '/src/main.js', format: 'TEXT', content: 'x' }; + mockFiles(outside, MAIN_JS); + + await callTool({ operations: [remove('/src/main.js', sha256Prefix('x'))] }); + + expect(getPutFiles()).toStrictEqual([MAIN_JS]); + }); + + it('edits and deletes stored files with a backslash in the path, so such files can be fixed or removed', async () => { + const util = { name: 'src\\util.js', format: 'TEXT', content: 'x' }; + const dockerfile = { name: '.\\Dockerfile', format: 'TEXT', content: 'FROM node' }; + mockFiles(dockerfile, util, MAIN_JS); + + await callTool({ + operations: [ + edit('src\\util.js', { oldText: 'x', newText: 'y' }), + remove('.\\Dockerfile', sha256Prefix('FROM node')), + ], + }); + + expect(getPutFiles()).toStrictEqual([MAIN_JS, { name: 'src\\util.js', format: 'TEXT', content: 'y' }]); + }); + + it('matches a path the way get-actor-version lists it', async () => { + const result = await callTool({ + operations: [{ type: 'delete', path: './src//main.js', expectedHash: MAIN_JS_HASH }], + }); + + expect(getPutFiles()).toStrictEqual([FOLDER, ACTOR_JSON, LOGO]); + expect(result.structuredContent.changes).toEqual([{ path: 'src/main.js', action: 'deleted' }]); + }); + + it.each([ + [write('a.js', 'a', { content: undefined }), 'content'], + [{ type: 'edit', path: 'a.js' }, 'edits'], + [{ type: 'delete', path: 'a.js' }, 'expectedHash'], + ])('refuses %j without the field its type needs', async (operation, field) => { + const text = await callToolExpectingUserError({ operations: [operation] }); + + expect(text).toBe(`operations[0] (${operation.type} a.js) needs ${field}.`); + }); + + it('writes nothing when the third operation fails', async () => { + const text = await callToolExpectingUserError({ + autoBuild: true, + operations: [ + write('src/a.js', 'a'), + edit('src/main.js', { oldText: 'const a = 1;', newText: 'const a = 2;' }), + { type: 'delete', path: 'src/none.js', expectedHash: MAIN_JS_HASH }, + ], + }); + + expect(text).toContain( + 'Nothing was written: operations[2] (delete src/none.js) failed with FILE_NOT_FOUND.', + ); + expect(buildMock).not.toHaveBeenCalled(); + }); + + const failingOperations = [ + { reason: 'FILE_EXISTS', operation: write('src/main.js', 'x') }, + { reason: 'FILE_NOT_FOUND', operation: remove('src/none.js', MAIN_JS_HASH) }, + { reason: 'HASH_MISMATCH', operation: remove('src/main.js', '0000000000000000') }, + { reason: 'NO_MATCH', operation: edit('src/main.js', { oldText: 'zzz', newText: 'y' }) }, + { reason: 'MULTIPLE_MATCHES', operation: edit('src/main.js', { oldText: 'a', newText: 'b' }) }, + { reason: 'NOT_TEXT', operation: edit('assets/logo.png', { oldText: 'PNG', newText: 'x' }) }, + ]; + it.each( + failingOperations.flatMap(({ reason, operation }) => + ['first', 'middle', 'last'].map((position, index) => ({ reason, operation, position, index })), + ), + )( + 'writes nothing and starts no build when the $position operation fails with $reason', + async ({ reason, operation, index }) => { + const operations: Record[] = [ + write('src/a.js', 'a'), + edit('.actor/actor.json', { oldText: '1', newText: '2' }), + ]; + operations.splice(index, 0, operation); + + const text = await callToolExpectingUserError({ autoBuild: true, operations }); + + const prefix = `${failedWith(index, operation.type, operation.path, reason)} `; + expect(text.startsWith(prefix), text).toBe(true); + expect(actorGetMock).toHaveBeenCalledTimes(1); + expect(versionMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }, + ); + + it('writes nothing and starts no build when expectedRevision does not match', async () => { + const text = await callToolExpectingUserError({ + autoBuild: true, + expectedRevision: 'aaaaaaaaaaaaaaaa', + operations: [write('b.js', 'b')], + }); + + expect(text).toContain('Nothing was written: expectedRevision failed with REVISION_MISMATCH.'); + expect(versionMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('reports REVISION_MISMATCH before an operation that would fail', async () => { + const revision = await readRevision(); + + const text = await callToolExpectingUserError({ + expectedRevision: 'aaaaaaaaaaaaaaaa', + operations: [write('src/main.js', 'x'), remove('src/none.js', MAIN_JS_HASH)], + }); + + expect(text).toBe( + 'Nothing was written: expectedRevision failed with REVISION_MISMATCH. ' + + `The version's revision is ${revision}, not aaaaaaaaaaaaaaaa.`, + ); + }); + + it('sends no PUT for no operations', async () => { + const result = await callTool({ operations: [] }); + + expect(versionUpdateMock).not.toHaveBeenCalled(); + expect(result.structuredContent).toEqual({ revision: await readRevision(), changed: false, changes: [] }); + // Nothing about runs: the stored files may already be in the build they use. + expect(result.content[1].text).toBe( + 'Nothing changed in version 0.1 of john/my-actor, so nothing was written.\nNo build was started.', + ); + }); + + it.each([ + [ + 'edits that undo each other', + [ + edit('src/main.js', { oldText: 'const a = 1;', newText: 'const a = 2;' }), + edit('src/main.js', { oldText: 'const a = 2;', newText: 'const a = 1;' }), + ], + ], + [ + 'a delete and a write of the same content', + [remove('src/main.js', MAIN_JS_HASH), write('src/main.js', MAIN_JS.content)], + ], + ['a write and a delete of a new file', [write('src/tmp.js', 'x'), remove('src/tmp.js', sha256Prefix('x'))]], + ])('sends no PUT for %s', async (_, operations) => { + const result = await callTool({ operations }); + + expect(versionUpdateMock).not.toHaveBeenCalled(); + expect(result.structuredContent).toEqual({ revision: await readRevision(), changed: false, changes: [] }); + }); + + it('refuses a stale expectedRevision (REVISION_MISMATCH) and takes a current one in either case', async () => { + const revision = await readRevision(); + const text = await callToolExpectingUserError({ + expectedRevision: 'aaaaaaaaaaaaaaaa', + operations: [write('b.js', 'b')], + }); + await callTool({ expectedRevision: revision.toUpperCase(), operations: [write('b.js', 'b')] }); + + expect(text).toBe( + 'Nothing was written: expectedRevision failed with REVISION_MISMATCH. ' + + `The version's revision is ${revision}, not aaaaaaaaaaaaaaaa.`, + ); + expect(versionUpdateMock).toHaveBeenCalledTimes(1); + }); + + it('refuses the first 8 characters of the revision, since it must match the whole revision (REVISION_MISMATCH)', async () => { + const revision = await readRevision(); + const expectedRevision = revision.slice(0, 8); + + const text = await callToolExpectingUserError({ + autoBuild: true, + expectedRevision, + operations: [write('b.js', 'b')], + }); + + expect(text).toBe( + 'Nothing was written: expectedRevision failed with REVISION_MISMATCH. ' + + `The version's revision is ${revision}, not ${expectedRevision}.`, + ); + expect(buildMock).not.toHaveBeenCalled(); + }); + + const hashPrefix = MAIN_JS_HASH.slice(0, 8); + it.each([ + ['write', write('src/main.js', 'x', { expectedHash: hashPrefix })], + ['delete', remove('src/main.js', hashPrefix)], + ])( + "refuses a %s given the first 8 characters of the file's hash, since it must match the whole hash (HASH_MISMATCH)", + async (type, operation) => { + const text = await callToolExpectingUserError({ autoBuild: true, operations: [operation] }); + + expect(text).toBe( + `${failedWith(0, type, 'src/main.js', 'HASH_MISMATCH')} ` + + `src/main.js has hash ${MAIN_JS_HASH}, not ${hashPrefix}.`, + ); + expect(buildMock).not.toHaveBeenCalled(); + }, + ); + + it('takes a null versionNumber, expectedRevision, and expectedHash as not given', async () => { + const args = { + actor: 'john/my-actor', + versionNumber: null, + expectedRevision: null, + operations: [write('b.js', 'b', { expectedHash: null })], + }; + + expect((updateActorVersion as HelperTool).ajvValidate(args)).toBe(true); + const result = await callTool(args); + + expect(versionMock).toHaveBeenCalledWith('0.1'); + expect(result.structuredContent.changes).toEqual([ + { path: 'b.js', action: 'created', hash: sha256Prefix('b') }, + ]); + }); + + it.each([ + [ + 'write', + write('src/main.js', 'x', { expectedHash: null }), + `${failedWith(0, 'write', 'src/main.js', 'FILE_EXISTS')} ` + + `src/main.js exists with hash ${MAIN_JS_HASH}; a write with that hash as expectedHash replaces it.`, + ], + [ + 'delete', + { type: 'delete', path: 'src/main.js', expectedHash: null }, + 'operations[0] (delete src/main.js) needs expectedHash.', + ], + ])( + 'refuses a %s of an existing file with a null expectedHash as one without it', + async (_, operation, message) => { + const args = { actor: 'john/my-actor', operations: [operation] }; + + expect((updateActorVersion as HelperTool).ajvValidate(args)).toBe(true); + const text = await callToolExpectingUserError(args); + + expect(text).toBe(message); + }, + ); + + it('computes the revision over the files only', async () => { + const result = await callTool({ operations: [] }); + + expect(result.structuredContent.revision).toBe( + buildFilesRevision([ + { path: '.actor/actor.json', hash: sha256Prefix(ACTOR_JSON.content) }, + { path: 'src/main.js', hash: MAIN_JS_HASH }, + { path: 'assets/logo.png', hash: sha256Prefix(LOGO_BYTES) }, + ]), + ); + }); + }); + + describe('Actor and version', () => { + it('needs versionNumber when the Actor has several versions', async () => { + actorGetMock.mockResolvedValue({ + id: 'actor-1', + name: 'my-actor', + username: 'john', + versions: [mockVersion(), mockVersion({ versionNumber: '0.2', buildTag: 'beta' })], + }); + + const text = await callToolExpectingUserError({ operations: [write('a.js', 'a')] }); + + expect(text).toBe( + 'Specify versionNumber; this Actor has versions: 0.1 (SOURCE_FILES, build tag latest), 0.2 (SOURCE_FILES, build tag beta).', + ); + }); + + it('lists only the versions that have a number when versionNumber is needed', async () => { + actorGetMock.mockResolvedValue({ + id: 'actor-1', + name: 'my-actor', + username: 'john', + versions: [ + mockVersion(), + { sourceType: 'SOURCE_FILES', sourceFiles: [MAIN_JS] }, + mockVersion({ versionNumber: '0.2', sourceType: 'GIT_REPO', buildTag: undefined }), + ], + }); + + const text = await callToolExpectingUserError({ operations: [write('a.js', 'a')] }); + + expect(text).toBe( + 'Specify versionNumber; this Actor has versions: 0.1 (SOURCE_FILES, build tag latest), 0.2 (GIT_REPO).', + ); + }); + + it.each([ + ['no versions', []], + ['only a version without a number', [{ sourceType: 'SOURCE_FILES', sourceFiles: [MAIN_JS] }]], + ])('needs versionNumber when the Actor has %s', async (_, versions) => { + actorGetMock.mockResolvedValue({ id: 'actor-1', name: 'my-actor', username: 'john', versions }); + + const text = await callToolExpectingUserError({ autoBuild: true, operations: [write('a.js', 'a')] }); + + expect(text).toBe('Specify versionNumber; this Actor has versions: .'); + expect(versionMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it.each(['qGXMy0NAkWsIIb9LZ', 'john~my-actor'])( + 'names the Actor by the full name in its document when given as %s', + async (actor) => { + const result = await callTool({ actor, operations: [write('a.js', 'a')] }); + + expect(actorMock.mock.calls).toEqual([[actor], ['actor-1']]); + expect(result.content[1].text).toBe( + 'Updated version 0.1 of john/my-actor.\nNo build was started; runs use these files only after the version is built.', + ); + }, + ); + + it('refuses a version the Actor does not have', async () => { + const text = await callToolExpectingUserError({ versionNumber: '0.2', operations: [write('a.js', 'a')] }); + + expect(text).toBe("Actor 'john/my-actor' has no version 0.2; available versions: 0.1."); + }); + + it('reports a missing Actor', async () => { + actorGetMock.mockResolvedValue(undefined); + + const text = await callToolExpectingUserError({ actor: 'my-actor', operations: [write('a.js', 'a')] }); + + expect(actorMock.mock.calls).toEqual([['~my-actor']]); + expect(text).toBe( + "Actor 'my-actor' not found. The actor argument takes an ID, a name (resolved against your own Actors), or a full name as username/name.", + ); + }); + + it.each([ + ['a version', 'john/my-actor/versions/0.1', mockVersion()], + // Has a name but no username. + ['an env var', 'john/my-actor/versions/0.1/env-vars/API_KEY', { name: 'API_KEY', value: 'secret-value' }], + // Has a username but no name, as a run's default storage does. + ["a run's key-value store", 'john/my-actor/runs/last/key-value-store', { id: 'kvs-1', username: 'john' }], + ])( + 'reports %s, reached by extra path segments, as a missing Actor and writes nothing', + async (_, actor, document) => { + actorGetMock.mockResolvedValue(document); + + const result = await callTool({ actor, autoBuild: true, operations: [write('a.js', 'a')] }); + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe( + `Actor '${actor}' not found. The actor argument takes an ID, a name (resolved against your own Actors), or a full name as username/name.`, + ); + expect(versionMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + expect(JSON.stringify(result)).not.toContain('secret-value'); + }, + ); + + it.each([ + ['https://user:secret@github.com/john/repo.git', 'https://github.com/john/repo.git'], + ['http://john:secret-password@git.example.com/repo.git', 'http://git.example.com/repo.git'], + [ + 'ssh://deploy:secret@github.com/john/repo.git?x=secret#main', + 'ssh://deploy@github.com/john/repo.git#main', + ], + ['git@github.com:john/repo.git', 'git@github.com:john/repo.git'], + ['git@github.com:john/repo.git?token=secret', 'git@github.com:john/repo.git'], + ['git@github.com:john/repo.git?token=secret#main:src', 'git@github.com:john/repo.git#main:src'], + ])('refuses a GIT_REPO version at %s, naming its URL as %s', async (gitRepoUrl, cleanUrl) => { + mockVersionRead({ sourceType: 'GIT_REPO', gitRepoUrl }); + + const result = await callTool({ autoBuild: true, operations: [write('a.js', 'a')] }); + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe( + `Version 0.1 of john/my-actor has its files in the Git repository ${cleanUrl}, not stored on Apify, so this tool cannot work on them; use the repository.`, + ); + expect(JSON.stringify(result)).not.toMatch(/secret/); + expect(versionMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it.each<{ outcome: string; version: Record; text: string }>([ + { + outcome: 'a GITHUB_GIST version, naming its URL without credentials', + version: { + sourceType: 'GITHUB_GIST', + gitHubGistUrl: 'https://gist.github.com/john/abc123?secret=x', + gitRepoUrl: 'https://github.com/old.git', + }, + text: 'Version 0.1 of john/my-actor has its files in the GitHub gist https://gist.github.com/john/abc123, not stored on Apify, so this tool cannot work on them; use the gist.', + }, + { + outcome: 'a TARBALL version without naming its URL', + version: { + sourceType: 'TARBALL', + tarballUrl: + 'https://api.example.test/v2/key-value-stores/s/records/version-0.1.zip?signature=secret', + }, + text: 'Version 0.1 of john/my-actor is stored as a zip archive (apify push does this for sources over 3 MiB), and this tool cannot work on zip-stored versions yet.', + }, + ])('refuses $outcome', async ({ version, text }) => { + mockVersionRead(version); + + const result = await callTool({ autoBuild: true, operations: [write('a.js', 'a')] }); + + expectSoftFailInvalidInput(result); + expect(result.content[0].text).toBe(text); + expect(JSON.stringify(result)).not.toMatch(/secret|example|old\.git|key-value-stores/); + expect(versionMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it.each(['SOURCE_CODE', 'SOMETHING_NEW'])( + 'refuses a %s version without naming a URL left over from another source type', + async (sourceType) => { + mockVersionRead({ + sourceType, + gitRepoUrl: 'https://github.com/john/old-repo.git', + tarballUrl: 'https://example.com/old-tarball.zip', + gitHubGistUrl: 'https://gist.github.com/john/old-gist', + }); + + const text = await callToolExpectingUserError({ autoBuild: true, operations: [write('a.js', 'a')] }); + + expect(text).toBe( + `Version 0.1 of john/my-actor has source type ${sourceType}, which this tool cannot work on; only versions stored as files are supported.`, + ); + expect(buildMock).not.toHaveBeenCalled(); + }, + ); + + it('refuses a version whose source the API hides', async () => { + mockVersionRead({ sourceFiles: undefined }); + + const text = await callToolExpectingUserError({ autoBuild: true, operations: [write('a.js', 'a')] }); + + expect(text).toBe( + "Version 0.1 of john/my-actor came back without its source: the API hides it from accounts that cannot modify the Actor. Ask the Actor's owner for the source.", + ); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it.each<[string, () => void, Record]>([ + ['a write', () => {}, { operations: [write('a.js', 'a')] }], + ['a failed operation', () => {}, { operations: [write('src/main.js', 'x')] }], + ['a build', () => {}, { autoBuild: true, operations: [write('a.js', 'a')] }], + ['a refused version', () => mockVersionRead({ sourceType: 'GIT_REPO' }), { operations: [] }], + ])('returns no env var name or value after %s', async (_, setup, args) => { + setup(); + + const result = await callTool(args); + + expect(JSON.stringify(result)).not.toMatch(/API_KEY|secret-value/); + }); + + it('lets a 4xx API error from the first attempt of the PUT through unchanged', async () => { + const error = apiError(400, 'Source files are too large.'); + versionUpdateMock.mockRejectedValue(error); + + await expect(callTool({ operations: [write('b.js', 'b')] })).rejects.toBe(error); + }); + + it('lets a 5xx API error from the read through unchanged, since nothing was sent', async () => { + const error = apiError(503, 'Service unavailable'); + actorGetMock.mockRejectedValue(error); + + await expect(callTool({ operations: [write('b.js', 'b')] })).rejects.toBe(error); + expect(versionUpdateMock).not.toHaveBeenCalled(); + }); + }); + + describe('a PUT that failed with a 5xx, without an HTTP response, or after a retry', () => { + // apify-client retries such a PUT, and the platform may have saved an attempt it did not answer. + const failures: [string, () => Error, string, Record][] = [ + [ + 'a 5xx', + () => apiError(503, 'Service unavailable', 'service-unavailable'), + 'with HTTP 503: Service unavailable (API error type: service-unavailable)', + { failureHttpStatus: 503, failureDetail: 'Service unavailable' }, + ], + [ + 'a 429 on a retry', + () => apiError(429, 'Rate limit exceeded', 'rate-limit-exceeded', 2), + 'with HTTP 429: Rate limit exceeded (API error type: rate-limit-exceeded)', + { + toolStatus: TOOL_STATUS.SOFT_FAIL, + failureCategory: FAILURE_CATEGORY.INVALID_INPUT, + failureHttpStatus: 429, + failureDetail: 'Rate limit exceeded', + }, + ], + [ + 'a network error', + () => Object.assign(new Error('socket hang up'), { request: {}, config: {}, code: 'ECONNRESET' }), + 'without an HTTP response: socket hang up', + { failureDetail: 'socket hang up' }, + ], + ]; + + it.each(failures)( + 'says after %s that the platform may have saved the change, which revision shows whether it did, and that no build was started', + async (_, buildError, failure, telemetry) => { + const previousRevision = await readRevision(); + versionUpdateMock.mockRejectedValue(buildError()); + + const result = await callTool({ autoBuild: true, operations: [write('b.js', 'b')] }); + // What get-actor-version returns once the platform stored the PUT. + mockFiles(...getPutFiles()); + const savedRevision = await readRevision(); + + expect(result.isError).toBe(true); + expect(result.toolTelemetry).toStrictEqual({ + toolStatus: TOOL_STATUS.FAILED, + failureCategory: FAILURE_CATEGORY.INTERNAL_ERROR, + ...telemetry, + }); + expect(result.structuredContent).toBeUndefined(); + expect(result.content).toStrictEqual([ + { + type: 'text', + text: + `Saving version 0.1 of john/my-actor failed ${failure}. The platform may still have saved the change. ` + + `If the version's revision is ${savedRevision}, this call saved it, and if it is ${previousRevision}, ` + + 'it saved nothing; get-actor-version reads the revision. No build was started.', + }, + ]); + expect(savedRevision).not.toBe(previousRevision); + expect(buildMock).not.toHaveBeenCalled(); + }, + ); + + it.each(failures)('names get-actor-version after %s only when the session has it', async (_, buildError) => { + versionUpdateMock.mockRejectedValue(buildError()); + const loadedToolNames = Object.values(HELPER_TOOLS).filter( + (name) => name !== HELPER_TOOLS.ACTOR_VERSION_GET, + ); + const context = stubToolCallContext( + { actor: 'john/my-actor', operations: [write('b.js', 'b')] }, + stubClient, + { loadedToolNames }, + ); + + const result = (await (updateActorVersion as HelperTool).call(context)) as UpdateResult; + + expect(result.isError).toBe(true); + expect(result.content[0].text).toMatch(/, and if it is [0-9a-f]+, it saved nothing\.$/); + expect(result.content[0].text).not.toContain(HELPER_TOOLS.ACTOR_VERSION_GET); + }); + }); + + describe('cancellation', () => { + it('writes nothing when the request is cancelled during the read', async () => { + const controller = new AbortController(); + actorGetMock.mockImplementation(async () => { + controller.abort(); + return { id: 'actor-1', name: 'my-actor', username: 'john', versions: [mockVersion()] }; + }); + + const result = await callTool({ autoBuild: true, operations: [write('b.js', 'b')] }, controller.signal); + + expect(result).toEqual({}); + expect(versionUpdateMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('starts no build when the request is cancelled during the write', async () => { + const controller = new AbortController(); + versionUpdateMock.mockImplementation(async () => controller.abort()); + + const result = await callTool({ autoBuild: true, operations: [write('b.js', 'b')] }, controller.signal); + + expect(result).toEqual({}); + expect(versionUpdateMock).toHaveBeenCalledTimes(1); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('keeps the write and starts no build when the request is cancelled during the Console link lookup', async () => { + const controller = new AbortController(); + vi.mocked(getUserInfoCached).mockImplementationOnce(async () => { + controller.abort(); + return mockUserInfo(); + }); + + const result = await callToolInConsole( + { autoBuild: true, operations: [write('b.js', 'b')] }, + controller.signal, + ); + + expect(result).toEqual({}); + expect(getUserInfoCached).toHaveBeenCalledTimes(1); + expect(versionUpdateMock).toHaveBeenCalledTimes(1); + expect(buildMock).not.toHaveBeenCalled(); + }); + + it('keeps the build it started when the request is cancelled during the build start', async () => { + const controller = new AbortController(); + buildMock.mockImplementation(async () => { + controller.abort(); + return { + id: 'build-1', + actId: 'actor-1', + buildNumber: '0.1.5', + status: 'RUNNING', + startedAt: new Date('2026-09-01T10:00:00.000Z'), + }; + }); + // The build client is what aborts a build. + const abortMock = vi.fn(async () => ({})); + buildClientMock.mockReturnValueOnce({ abort: abortMock }); + + const result = await callTool({ autoBuild: true, operations: [write('b.js', 'b')] }, controller.signal); + + expect(versionUpdateMock).toHaveBeenCalledTimes(1); + expect(buildMock).toHaveBeenCalledTimes(1); + expect(buildClientMock).not.toHaveBeenCalled(); + expect(abortMock).not.toHaveBeenCalled(); + expect(result.structuredContent.build).toEqual({ + id: 'build-1', + actorId: 'actor-1', + buildNumber: '0.1.5', + status: 'RUNNING', + startedAt: '2026-09-01T10:00:00.000Z', + finishedAt: null, + }); + }); + }); + + describe('autoBuild', () => { + it('starts a build without waiting', async () => { + const result = await callTool({ autoBuild: true, operations: [write('b.js', 'b')] }); + + expectSchemaConformingStructuredContent(result, updateActorVersionToolOutputSchema); + expect(buildMock).toHaveBeenCalledWith('0.1', { useCache: true }); + expect(result.structuredContent.build).toEqual({ + id: 'build-1', + actorId: 'actor-1', + buildNumber: '0.1.5', + status: 'READY', + startedAt: '2026-09-01T10:00:00.000Z', + finishedAt: null, + }); + expect(result.content[1].text).toContain(`Check progress with ${HELPER_TOOLS.ACTOR_BUILD_GET}`); + expect(result.content).toHaveLength(2); + expect(JSON.parse(result.content[0].text)).toEqual(result.structuredContent); + }); + + it('says the build is still running without naming get-actor-build when the session lacks it', async () => { + const loadedToolNames = Object.values(HELPER_TOOLS).filter((name) => name !== HELPER_TOOLS.ACTOR_BUILD_GET); + const context = stubToolCallContext( + { actor: 'john/my-actor', autoBuild: true, operations: [write('b.js', 'b')] }, + stubClient, + { loadedToolNames }, + ); + + const result = (await (updateActorVersion as HelperTool).call(context)) as UpdateResult; + + expect(buildMock).toHaveBeenCalledWith('0.1', { useCache: true }); + expect(result.content[1].text).toBe( + 'Updated version 0.1 of john/my-actor.\nThe build is still running; check its status again in a few seconds.', + ); + }); + + it('adds the build Console link for an Apify Console session', async () => { + vi.mocked(getUserInfoCached).mockResolvedValue(mockUserInfo()); + + const result = await callToolInConsole({ autoBuild: true, operations: [write('b.js', 'b')] }); + + const consoleUrl = 'https://console.apify.com/actors/actor-1/builds/0.1.5'; + expectSchemaConformingStructuredContent(result, updateActorVersionToolOutputSchema); + expect(getUserInfoCached).toHaveBeenCalledWith('apify_ui_test', stubClient); + expect(result.structuredContent.build?.apifyConsoleUrl).toBe(consoleUrl); + expect(result.content).toHaveLength(3); + expect(JSON.parse(result.content[0].text)).toEqual(result.structuredContent); + expect(result.content[2].text).toBe(`Apify Console: ${consoleUrl}\n${VERBATIM_LINKS_NUDGE}`); + }); + + it.each([ + ['no operations', []], + [ + 'a write that leaves the content as it was', + [write('src/main.js', MAIN_JS.content, { expectedHash: MAIN_JS_HASH })], + ], + ])('starts no build for %s, and says nothing changed', async (_, operations) => { + vi.mocked(getUserInfoCached).mockResolvedValue(mockUserInfo()); + + const result = await callToolInConsole({ autoBuild: true, operations }); + + expectSchemaConformingStructuredContent(result, updateActorVersionToolOutputSchema); + expect(versionUpdateMock).not.toHaveBeenCalled(); + expect(buildMock).not.toHaveBeenCalled(); + expect(getUserInfoCached).not.toHaveBeenCalled(); + expect(result.structuredContent).toEqual({ revision: await readRevision(), changed: false, changes: [] }); + expect(result.content).toHaveLength(2); + expect(result.content[1].text).toBe( + 'Nothing changed in version 0.1 of john/my-actor, so nothing was written.\nNo build was started.', + ); + }); + + it('returns a build that is still running without waiting for it', async () => { + buildMock.mockResolvedValue({ id: 'build-2', actId: 'actor-1', buildNumber: '0.1.6', status: 'RUNNING' }); + + const result = await callTool({ autoBuild: true, operations: [write('b.js', 'b')] }); + + expect(result.structuredContent.build).toEqual( + expect.objectContaining({ id: 'build-2', status: 'RUNNING' }), + ); + expect(buildClientMock).not.toHaveBeenCalled(); + }); + + it('reports a build that failed to start with the write still done', async () => { + buildMock.mockRejectedValue(apiError(402, 'Not enough credit')); + + const result = await callTool({ autoBuild: true, operations: [write('b.js', 'b')] }); + + expectSchemaConformingStructuredContent(result, updateActorVersionToolOutputSchema); + expect(versionUpdateMock).toHaveBeenCalledTimes(1); + expect(result.structuredContent.buildError).toBe('Not enough credit'); + expect(result.structuredContent).not.toHaveProperty('build'); + expect(result.content[1].text).toContain( + 'The build could not be started; runs use these files only after the version is built.', + ); + }); + + it.each([ + [ + 'a network failure after the retries', + Object.assign(new Error('socket hang up'), { request: {}, config: {}, code: 'ECONNRESET' }), + ], + ['a failure that is not an API error', new TypeError('Cannot read properties of undefined')], + ])('reports %s of the build start as buildError, with the write still done', async (_, error) => { + buildMock.mockRejectedValue(error); + + const result = await callTool({ autoBuild: true, operations: [write('b.js', 'b')] }); + + expectSchemaConformingStructuredContent(result, updateActorVersionToolOutputSchema); + expect(versionUpdateMock).toHaveBeenCalledTimes(1); + expect(result.structuredContent).toEqual({ + revision: expect.stringMatching(/^[0-9a-f]{16}$/), + changed: true, + changes: [{ path: 'b.js', action: 'created', hash: sha256Prefix('b') }], + buildError: error.message, + }); + expect(result.content[1].text).toBe( + 'Updated version 0.1 of john/my-actor.\nThe build could not be started; runs use these files only after the version is built.', + ); + }); + + it('reports a failed Console link lookup as buildError before any build starts, with the write still done', async () => { + vi.mocked(getUserInfoCached).mockRejectedValue(new Error('users/me failed')); + + const result = await callToolInConsole({ autoBuild: true, operations: [write('b.js', 'b')] }); + + expect(versionUpdateMock).toHaveBeenCalledTimes(1); + expect(buildMock).not.toHaveBeenCalled(); + expect(result.structuredContent.buildError).toBe('users/me failed'); + expect(result.structuredContent.changes).toEqual([ + { path: 'b.js', action: 'created', hash: sha256Prefix('b') }, + ]); + }); + }); +}); diff --git a/tests/unit/utils.auth.test.ts b/tests/unit/utils.auth.test.ts index 2e724e2c9..205a5b706 100644 --- a/tests/unit/utils.auth.test.ts +++ b/tests/unit/utils.auth.test.ts @@ -2,7 +2,7 @@ import { describe, expect, it } from 'vitest'; import { isApiTokenRequired } from '../../src/utils/auth.js'; -const RETIRED_SELECTORS = ['add-actor', 'experimental', 'preview', 'get-actor-log'] as const; +const RETIRED_SELECTORS = ['add-actor', 'experimental', 'preview', 'get-actor-log', 'source'] as const; describe('isApiTokenRequired', () => { it('requires token if no tools are specified', () => { diff --git a/tests/unit/utils.tools_loader.retired_selectors.test.ts b/tests/unit/utils.tools_loader.retired_selectors.test.ts index 69e9ae66c..a0793a798 100644 --- a/tests/unit/utils.tools_loader.retired_selectors.test.ts +++ b/tests/unit/utils.tools_loader.retired_selectors.test.ts @@ -8,7 +8,7 @@ import type * as ToolsIndexModule from '../../src/tools/index.js'; import { SERVER_MODE } from '../../src/types.js'; import { getActors, getToolsForServerMode, toolNamesToInput } from '../../src/utils/tools_loader.js'; -const RETIRED_SELECTORS = ['add-actor', 'experimental', 'preview', 'get-actor-log'] as const; +const RETIRED_SELECTORS = ['add-actor', 'experimental', 'preview', 'get-actor-log', 'source'] as const; vi.mock('../../src/tools/index.js', async (importOriginal) => { const actual = await importOriginal();