Skip to content

feat(chip-testing): add AllDevicesTestApp mimic for spec-compliant per-device-type tests#3687

Merged
Apollon77 merged 40 commits intomainfrom
feat/all-devices-test-app
May 6, 2026
Merged

feat(chip-testing): add AllDevicesTestApp mimic for spec-compliant per-device-type tests#3687
Apollon77 merged 40 commits intomainfrom
feat/all-devices-test-app

Conversation

@Apollon77
Copy link
Copy Markdown
Collaborator

@Apollon77 Apollon77 commented May 4, 2026

Summary

  • Adds AllDevicesTestApp, a new support/chip-testing mimic that mirrors connectedhomeip's examples/all-devices-app — code-driven endpoints, runtime device-type selection via repeatable --device <type[:endpoint]>, optional --wifi. CLI surface is 1:1 with chip's binary.
  • Implements all 9 device types chip's all-devices-app supports today (now possible because Matter 1.5.1 just landed on main): chime, contact-sensor, dimmable-light, occupancy-sensor, on-off-light, soil-sensor, speaker, temperature-sensor, water-leak-detector. Each lives in its own file under support/chip-testing/src/devices/ and self-registers in a small DeviceTypeRegistry.
  • Switches all 10 --app-path all-devices: mappings in .github/workflows/chip-tool-tests.yml from AllClustersTestApp.{js,sh} to AllDevicesTestApp.{js,sh} so the existing all-devices test target now runs against the new spec-compliant mimic instead of aliasing AllClusters. chip-matterjs-tests.yml:113 (chip-native bin/all-devices-app) is left untouched.

What stays the same

  • AllClustersTestApp is unchanged. The all-clusters test target keeps using it.
  • RootEndpoint.ts is a verbatim lift of AllClustersTestInstance.setupServer() lines 264-377 with two declared seams: WiFi/Ethernet NetworkCommissioning variant + enable-key source. All chip-test workarounds (CADMIN/1.15 + SC/4.3 broadcast schedule, Buddhist calendar, TCP transport flags) are preserved so AllDevices inherits the same chip-test compatibility AllClusters has today.
  • BackchannelCommand.endpointId and command.newState field names match the existing AllClustersTestInstance handler.

File layout

support/chip-testing/
  src/
    AllDevicesTestApp.ts             # bin entrypoint
    AllDevicesTestInstance.ts        # orchestrator (CLI parse, setupServer, backchannel dispatch)
    GenericTestApp.ts                # +getParameters helper for repeatable flags
    devices/
      DeviceTypeRegistry.ts
      RootEndpoint.ts                # EP0 builder with WiFi/Ethernet seam
      all-devices.ts                 # registration barrel
      {Chime,ContactSensor,DimmableLight,OccupancySensor,OnOffLight,SoilSensor,Speaker,TemperatureSensor,WaterLeakDetector}Endpoint.ts
  AllDevicesTestApp.sh               # SIGTERM-handling shell wrapper

Notes

  • --device is required; missing/unknown types fail loud with the supported list — chip parity.
  • Endpoint number defaults to 1 and auto-increments, skipping any number reserved by an explicit :N. CLI order is preserved.
  • --wifi swaps NetworkCommissioningServer.with("EthernetNetworkInterface") for WiFiNetworkInterface and seeds supportedWiFiBands: [2G4] plus minimal scan/connect timeouts so behavior validation passes.
  • WiFi command implementations (scanNetworks, addOrUpdateWiFiNetwork, etc.) are not provided — chip tests probing those will fail, which is the expected v1 stance. A follow-up can add stub command handlers if a test target needs them.
  • command.endpointId filtering in stateful device backchannels (contact-sensor, water-leak-detector) routes correctly when multiple boolean-state-style devices share an instance.

🤖 Generated with Claude Code

Apollon77 and others added 21 commits May 4, 2026 15:38
Adds getParameters(name): string[] alongside existing getParameter/hasParameter/getIntParameter
in GenericTestApp.ts. Returns all values for repeated -name <v> / --name <v> occurrences in
argv order. Required by the upcoming AllDevicesTestApp's repeatable --device flag.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces EndpointHandle / DeviceTypeEntry interfaces and a Map-based registry
with register/get/list helpers. Forms the seam between AllDevicesTestInstance
(orchestrator) and per-device-type endpoint modules (each self-registers via
side-effect import).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lifts AllClustersTestInstance.setupServer()'s ServerNode.create body verbatim into
buildRootNode(opts), exposing two seams:
  1. NetworkCommissioning variant + networkCommissioning.networks switch on opts.wifi
     (Ethernet -> WiFi stub with empty networks list).
  2. enableKeyHex sourced from opts instead of process.argv parse.

All other root behaviors, basicInformation, productDescription, MDNS broadcast schedule,
generalDiagnostics, localizationConfiguration, networkCommissioning details,
operationalCredentials, timeFormatLocalization (incl. Buddhist calendar workaround),
and userLabel are preserved exactly so AllDevicesTestApp inherits the same chip-test
compatibility AllClustersTestApp has today.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First per-device-type endpoint module. Self-registers in DeviceTypeRegistry
under the chip-CLI-compatible name "on-off-light" and constructs a single
OnOffLightDevice endpoint. No backchannel — the spec doesn't require any
test-injected commands for plain on/off lights.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same pattern as on-off-light: plain DimmableLightDevice on the requested endpoint,
no backchannel. Spec compliance comes from the device-type definition itself.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…te backchannel

Initial booleanState.stateValue = false. Backchannel handles the same
setBooleanState command that AllClustersTestInstance.backchannel() processes today
(field name command.newState matches AllClusters convention). Endpoint-id filter
prevents accidental cross-talk when multiple boolean-state-style devices share an
AllDevicesTestApp instance.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…anState backchannel

Mirrors ContactSensorEndpoint shape but uses WaterLeakDetectorDevice so the device
type identity matches the chip CLI string "water-leak-detector". Same setBooleanState
backchannel — chip's harness sends the same command shape regardless of which boolean-
state-style sensor is targeted; the endpoint-id filter routes correctly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
OccupancySensorDevice requires explicit OccupancySensingServer feature selection per
Matter 1.5 cluster definition; we pick PassiveInfrared (matches AllClustersTestInstance
EP1 wiring). Initial occupied=false to mirror chip's TogglingOccupancySensorDevice
baseline. No backchannel — AllClusters has no precedent handler for occupancy.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Initial measuredValue 2500 (25.00 degC, centi-degrees). No backchannel — the
BackchannelCommand union does not currently include a setTemperature variant and
AllClustersTestInstance has no precedent handler. A handler can be added later once
the union is extended in @matter/testing.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sApp

Side-effect imports trigger each per-device module's top-level registerDeviceType
call so the registry is populated by the time AllDevicesTestInstance.setupServer
runs. Alphabetical order. Adding a future device type (chime, soil-sensor, speaker
once Matter 1.5.1 lands) is a one-line edit here plus a new module file.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends NodeTestInstance with:
  - parseDeviceArgs() — two-pass parser that respects CLI order while honoring
    explicit :N endpoint reservations. Spec example
    `--device contact-sensor --device on-off-light:5 --device occupancy-sensor`
    produces [contact-sensor:EP1, on-off-light:EP5, occupancy-sensor:EP2].
  - setupServer() — builds root via buildRootNode (with --wifi seam and
    --enable-key passthrough), then resolves each spec to a registered factory
    and adds the constructed endpoint. Throws ValidationError with sorted
    listDeviceTypes() in the message for missing/unknown --device flags.
  - backchannel() — targeted-then-broadcast dispatch. command.endpointId
    addresses one endpoint first; remaining endpoints get a broadcast pass;
    unhandled commands fall through to NodeTestInstance.backchannel.

Identity "alldevices-6100" keeps KVS namespace separate from AllClusters'
"binford-6100". Side-effect import of devices/all-devices.js triggers all
registerDeviceType calls before setupServer runs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors AllClustersTestApp.ts shape: imports @matter/nodejs platform, sets
process.title for stress-test detection, logs pid/argv for CI parsing, then
delegates to startDeviceTestApp(AllDevicesTestInstance, StorageBackendAsyncJsonFile).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SIGTERM-handling wrapper sibling to AllClustersTestApp.sh. Tees stdout/stderr to
test_alldevices.log, normalizes exit code 134 (abort) to 0 on SIGTERM, forwards
all args to dist/esm/AllDevicesTestApp.js. Used by chip-tool-tests.yml jobs that
expect a script entry rather than a node invocation.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
….5.1)

ChimeServer requires installedChimeSounds (non-empty), selectedChime matching
one of them, and enabled. Defaults to a single "Default" sound (chimeId=0)
with playback enabled — sufficient for chip's chime test surface; tests that
exercise sound selection can write a different selectedChime.

Closes follow-up filed during initial design (TODO under matter-1.5/all-devices-app-chime.md).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…tter 1.5.1)

SoilMeasurement cluster requires both soilMoistureMeasurementLimits (a
MeasurementAccuracy struct with at least one accuracyRange entry) and
soilMoistureMeasuredValue. Defaults to a single 0-99% range with 5%
percentMax accuracy, initial measuredValue 50% — chip's all-devices-app
uses an IncreasingMoistureSoilSensorDevice that mutates this over time;
our mimic provides a static baseline.

Closes follow-up filed during initial design (TODO under matter-1.5/all-devices-app-soil-sensor.md).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… 1.5.1)

SpeakerDevice composes OnOffServer (mute) and LevelControlServer (volume) per
the Matter 1.5.1 spec. No initial state needed — cluster defaults are valid.
No backchannel — chip's LoggingSpeakerDevice in all-devices-app just logs;
test coverage exercises the cluster commands directly.

Closes follow-up filed during initial design (TODO under matter-1.5/all-devices-app-speaker.md).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…pp barrel

Adds the 3 newly implemented device types to the side-effect-import barrel so
they self-register at AllDevicesTestInstance startup. Matches chip's all-devices-app
supported set 1:1 (9 device types total).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… in AllDevices RootEndpoint

The 1.5.1 NetworkCommissioning cluster requires non-empty supportedWiFiBands
(min 1) plus scanMaxTimeSeconds and connectMaxTimeSeconds when WiFi feature is
selected. Without these, the WiFi-variant root fails behavior validation at
ServerNode.create with "Array length 0 is not within bounds" on supportedWiFiBands.
Add minimal stub values (2.4GHz band, 1s timeouts) so --wifi smoke passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Switches all 10 --app-path all-devices: mappings in chip-tool-tests.yml from
AllClustersTestApp.{js,sh} to AllDevicesTestApp.{js,sh}. The matter.js side
of the all-devices test target now exercises the spec-compliant per-device-type
mimic instead of the kitchen-sink AllClusters EP1 it shared with all-clusters.

all-clusters: mappings are untouched. chip-matterjs-tests.yml:113 (chip-native
bin/all-devices-app) is also untouched — that workflow tests matter.js controller
against chip's own device binary.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…dpoint

AllClustersTestInstance's network block sets `tcp: true` and a
TEST_PREFER_TCP-driven `transportPreference`. Without these, the new
AllDevicesTestApp silently runs UDP-only even when the matterjs-tests-core-tcp
CI job exports TEST_PREFER_TCP=1 — that job's `--app-path all-devices:` now
points at AllDevicesTestApp, so the bug would let TCP transport tests pass-by-skip
rather than actually exercising TCP.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The two TestSelfFabricRemoval invocations in chip-tool-tests.yml fall back to
cat ./test_allclusters.log on failure. Now that AllDevicesTestApp.sh writes
to ./test_alldevices.log and the all-devices test target maps to that script,
also cat ./test_alldevices.log so a failure originating from AllDevices is
debuggable in CI logs (otherwise we'd see the AllClusters log alone, likely
unrelated to the actual failure).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 4, 2026 14:59
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new support/chip-testing “AllDevicesTestApp” mimic (modeled after connectedhomeip’s all-devices-app) so chip-tool test runs can execute spec-compliant, per-device-type endpoints instead of aliasing to AllClusters.

Changes:

  • Introduces AllDevicesTestApp/AllDevicesTestInstance with repeatable --device <type[:endpoint]> parsing, device registry dispatch, and optional --wifi.
  • Adds per-device-type endpoint modules (9 device types) that self-register via a DeviceTypeRegistry, plus a shared EP0 RootEndpoint builder.
  • Updates chip-tool-tests.yml mappings so the all-devices test target runs the new app (and includes its log on failure for the .sh runner case).

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
support/chip-testing/src/GenericTestApp.ts Adds getParameters() helper to support repeatable CLI flags (used by --device).
support/chip-testing/src/AllDevicesTestApp.ts New Node entrypoint for the AllDevices mimic.
support/chip-testing/src/AllDevicesTestInstance.ts New orchestrator: parses --device, builds root node, registers endpoints, dispatches backchannel commands.
support/chip-testing/src/devices/DeviceTypeRegistry.ts Registry for device type factories + listing supported types.
support/chip-testing/src/devices/RootEndpoint.ts EP0 builder (largely copied from AllClusters) with WiFi/Ethernet seam and enable-key seam.
support/chip-testing/src/devices/all-devices.ts Side-effect import barrel that registers all supported device types.
support/chip-testing/src/devices/ChimeEndpoint.ts Adds “chime” device endpoint factory and registration.
support/chip-testing/src/devices/ContactSensorEndpoint.ts Adds “contact-sensor” endpoint factory + boolean-state backchannel handler.
support/chip-testing/src/devices/DimmableLightEndpoint.ts Adds “dimmable-light” device endpoint factory and registration.
support/chip-testing/src/devices/OccupancySensorEndpoint.ts Adds “occupancy-sensor” endpoint factory and registration (PIR feature).
support/chip-testing/src/devices/OnOffLightEndpoint.ts Adds “on-off-light” device endpoint factory and registration.
support/chip-testing/src/devices/SoilSensorEndpoint.ts Adds “soil-sensor” endpoint factory and registration (soil measurement config).
support/chip-testing/src/devices/SpeakerEndpoint.ts Adds “speaker” device endpoint factory and registration.
support/chip-testing/src/devices/TemperatureSensorEndpoint.ts Adds “temperature-sensor” endpoint factory and registration (measuredValue preset).
support/chip-testing/src/devices/WaterLeakDetectorEndpoint.ts Adds “water-leak-detector” endpoint factory + boolean-state backchannel handler.
support/chip-testing/AllDevicesTestApp.sh New SIGTERM-handling wrapper script + dedicated log file for the AllDevices mimic.
.github/workflows/chip-tool-tests.yml Switches all-devices app-path mappings to the new AllDevices app, and prints its log on wrapper-run failures.

Comment thread support/chip-testing/src/GenericTestApp.ts Outdated
Comment thread support/chip-testing/src/AllDevicesTestInstance.ts Outdated
Comment thread support/chip-testing/AllDevicesTestApp.sh Outdated
Apollon77 and others added 6 commits May 4, 2026 17:22
Per Copilot PR review: silently dropping a trailing -name/--name token (no
value following) makes "--device required" errors confusing when the user
actually typed --device. Now throws ValidationError with the parameter name,
matching chip CLI parity expectations.

Existing single-value getParameter retains its lenient behavior to avoid
churn in callers that rely on it; the strict throw applies only to the new
repeatable helper used by --device.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per Copilot PR review: parseDeviceArgs validated ep >= 1 but not ep <= 0xFFFE,
so --device foo:70000 would surface a generic out-of-bounds throw from the
EndpointNumber() brand at later use rather than the token-specific
"Invalid endpoint in --device" message. Add the upper-bound check next to
the existing lower-bound check.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…SIGTERM handler

Per Copilot PR review: when SIGTERM arrives before CHILD_PID is assigned, the
prior implementation captured \$? from the [ -n "\$CHILD_PID" ] test (status 1)
rather than from a wait, so the wrapper would exit 1 even though no child
failed. Initialize EXIT_CODE=0 up front and only overwrite from wait when a
child was actually waited for.

(The same bug exists in AllClustersTestApp.sh — left for a separate follow-up
to keep this change scoped.)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… SIGTERM handler

Same bug we just fixed in AllDevicesTestApp.sh: when SIGTERM arrives before
CHILD_PID is assigned, the prior code captured \$? from the [ -n "\$CHILD_PID" ]
test rather than wait, so the wrapper exited 1 with no child failure.
Initialize EXIT_CODE=0 up front and only overwrite from wait when a child was
actually waited for.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a second optional parameter Subject.Options to the Factory signature so
per-run app-args (chip header `app-args:`) can flow into in-process subjects
without going through process.argv. Existing factories that ignore the extra
arg remain compatible.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Introduces a Map<string, Subject.Factory> keyed by chip-test-header app names
("all-clusters", "all-devices", "bridge", "tv", "rvc", future). Setup-time
registration via chip.subjectFor("name", factory). Duplicate registration
throws. Task 3 will wire the registry into defineTest dispatch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Apollon77 and others added 12 commits May 5, 2026 08:59
…args

Wires the runtime side of the chip-test app registry:
  - parseAppArgs() splits descriptor.config["app-args"] (string or array form).
  - defineTest's beforeOne resolves the factory by priority: explicit .subject()
    override on the builder, then State.subjectForApp(descriptor.app), then
    chip.defaultSubject. A descriptor that names an unregistered app fails loud
    with a message naming the missing app and pointing at chip.subjectFor.
  - activateSubject + loadSubject thread appArgs through. The subject cache key
    becomes (factory, kind, appArgs) so distinct per-run app-args produce
    distinct cached subjects (chip multi-run tests with different --device flags).

Backward compatible: tests without descriptor.app keep using chip.defaultSubject.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds an optional appArgs?: string[] field to DeviceTestInstanceConfig so the
support.ts factory can forward chip-framework Subject.Options.appArgs into the
TestInstance constructor. Subjects that need runtime CLI-style configuration
(AllDevicesTestApp's --device list) consume it via this.config.appArgs in
setupServer(). Existing TestInstance classes (AllClusters/Bridge/Tv/Rvc) ignore
the field — change is purely additive.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… config first

Replaces parseDeviceArgs() (which read process.argv directly via getParameters/
hasParameter/getParameter) with parseRuntimeArgs(args[]) — same logic but
argv-source-independent. Constructor extracts config.appArgs into a private
field; setupServer() uses this.#appArgs ?? process.argv.slice(2) as the source.

Behavior unchanged for standalone CLI runs (chip-tool-tests CI binary, local
smoke). Test framework now flows per-run --device/--wifi/--enable-key into the
subject via Subject.Options.appArgs, unblocking chip multi-run python tests
where each run targets a different device type.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…criptor-app dispatch

Adds AllDevicesApp and registers all 5 subjects (all-clusters, all-devices,
bridge, tv, rvc) under their normalized chip-header names. The App() helper
now also threads Subject.Options.appArgs into the constructor so per-run
chip-header app-args (e.g. "--device on-off-light:1") flow into the in-process
subject. chip.defaultSubject stays at AllClustersApp for tests without
descriptor.app.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI regression: forwarding parseAppArgs() output for every test (including
tests that fall through to chip.defaultSubject) caused per-test cache key
divergence — chip headers carry per-test --trace-to / --KVS values, so
loadSubject() spawned a fresh subject for every test, thrashing MDNS and
producing CASE timeouts on subsequent tests (TC_WHM_2_1, TC_DRLK_*, etc).

Only set appArgs when descriptor.app is at top-level (= multi-run suite
member explicitly dispatched via chip.subjectFor). Tests using
chip.defaultSubject keep their pre-PR cache behavior (kind only).

Also enhances print-report to surface descriptor.app per test in brackets,
which made the bug easy to spot during inspect.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…or generator

chip's MatterBaseTest.get_test_desc(test) does getattr(self, test) and requires the
full method name (test_TC_FOO_2_3). The legacy short-form aliases ("FOO_2_3") that
made our previous call signature work have been removed upstream — TC_CGEN_2_4 now
raises AttributeError, which abort-aborts the descriptor build and leaves the chip
docker image with a stale /lib/test-descriptor.json (no recent multi-run / new app
references, no chime/soil/speaker, etc).

Pass test_method_name instead of the stripped form. Wrap in try/except so a single
bad test class can't break the whole image build going forward.

[rebuild-chip]

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ad of erroring

When a multi-run python test descriptor names an app we have no Subject.Factory
for (e.g. \${LOCK_APP}, \${LIT_ICD_APP}, \${WATER_HEATER_APP}), skip just that run
rather than throwing. Chip CI lists more apps than we mimic; whitelist semantics
let the runs we DO support proceed without surfacing failures from runs we
deliberately don't cover. Other runs of the same test continue normally.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Including appArgs in loadSubject's cache key spawned a fresh in-process subject
for each multi-run member of a chip python test that shared a factory but
differed in script-side flags (e.g. TC_SWTCH run1/run2/run3 all use
ALL_CLUSTERS_APP but pass different --app-pipe paths). Each new subject was
commissioned independently — chip controller storage held one fabric while the
restored subject snapshot held another, producing the symptom

  CHIP: Compressed FabricId 0x6BB913D815052317
  matter.js: mdns:074A2057D2EE4E7C-...

and CASE Avahi resolve timeouts.

Cache by (factory, kind) only. appArgs are still forwarded on first construct
(needed for AllDevicesTestApp's --device list); cache hits reuse the cached
subject's commissioning state so all multi-run members of a single factory
share fabric and chip controller storage stays consistent.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- support/chip/sha.txt: 6891456 (Apr 21) -> e8152e9 (today). The Apr 21 pin
  predated chip's TC_OO_2_7 multi-run addition (Apr 23) plus all the recent
  chime/soil/speaker/etc test additions, so our descriptor never picked them up.
- print-report.ts: surface descriptor.config.app in the [...] tag when
  descriptor.app (top-level, set only for multi-run suite members) is absent.
  Single-run python tests now also display the app they target.

[rebuild-chip]

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ddition

e8152e9 (today's master) added a step0c label to TC_LVL_9_1, which the matter.js
LevelControl behavior doesn't yet satisfy and surfaced as the only test-app-slow
failure on PR #3687. Roll the pin back one commit to ee82434 (also today, just
the prior commit) — keeps the multi-run TC_OO_2_7 (added Apr 23) and chime/soil/
speaker definitions, drops the new step0c assertion. TC_LVL_9_1 step0c is
follow-up matter.js work.

[rebuild-chip]

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a diagnostic-only test file (test/diag/LVL_9_1.test.ts), an
npm test-diag script, and a continue-on-error CI job in build-test.chip.yml.
The job runs only LVL/9.1 with --all-logs to capture the full chip subprocess
output so we can see why TC_LVL_9_1 fails against newer chip pins (annotation-
level CI logs swallow chip stderr/stdout).

[rebuild-chip]

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…try fix

Upstream chip landed the missing
\`if __name__ == "__main__": default_matter_test_main()\`
on TC_LVL_9_1.py, so the test now actually runs when invoked directly. Bump
the pin and trigger an image rebuild to verify.

[rebuild-chip]

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Apollon77 Apollon77 force-pushed the feat/all-devices-test-app branch from 6edd274 to 1f9ffc7 Compare May 5, 2026 20:52
Upstream chip added the missing default_matter_test_main() entry; the
test-diag job confirmed TC_LVL_9_1 runs against the new chip pin. Remove the
diagnostic test file, npm script, and CI job — served their purpose.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Apollon77 Apollon77 merged commit eae343f into main May 6, 2026
38 checks passed
@Apollon77 Apollon77 deleted the feat/all-devices-test-app branch May 6, 2026 06:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants