This web application is the official ADL LRS Test Suite. The Test Suite evaluates whether an LRS correctly implements the mandatory xAPI server-side requirements by automating HTTP requests to an LRS and evaluating its responses. The Test Suite evaluates over 1300 LRS testing requirements, which were derived from the xAPI specification as well as community input from the Policies and Procedures for Conformance Testing Group.

At the end of a successful test, the Test Suite creates an authoritative certificate verifying conformance. This allows organizations that are developing or acquiring an LRS to ensure it fully adheres to the xAPI specification, which aids developers, DoD and Federal agencies, and the broader community as they seek to implement xAPI-conformant systems.

Submitting Issues or Questions

The Test Suite team use the GitHub issues list for the lrs-conformance-test-suite project to collect any issues or questions about the LRS Test Suite and the xAPI Adopters Registry. This keeps all issues in one place, provides access to a public archive of historical data, and allows the community to comment and help answer issues.

Software Updates Schedule

We use a consistent release schedule to allow LRS developers to plan and prepare for additional LRS testing. This schedule will be on a monthly basis.

The Test Suite team will follow the update schedule described below:

1. On the 1st Wednesday of each month, the Test Suite team will stop adding new patches to the ‘staged’ branch.
   1. If there are no staged fixes by the 1st Wednesday of the month, there will be no software update for the current month.
2. Next, the Test Suite team will test the software and take community feedback over a review period of one week.
3. On the 2nd Wednesday of the month, the Test Suite team will merge ‘staged’ to ‘master' and push the changes to the LRS Test Suite production environment.

Bug Fix Process

Any issues found to be bugs in the user interface or the web application that do not impact testing and conformance will be patched and released to the public as they are fixed.

Any issues found to be bugs in a test case or in the testing software that may impact conformance, will follow the fix and release process outlined below:

1. Issues will be reviewed by the Test Suite team.
   1. Issues determined to be obvious bugs - wrong expected error code, etc - will be tasked out to team members to fix.
   2. Issues that do not have a clear or obvious fix will be brought up to the LRS Testing Group for a recommended resolution and consensus, then tasked out to team members to fix.
2. When issues are fixed the changes will be merged into the ‘staged’ branch of the lrs-conformance-test-suite project and the issue will be tagged as ‘staged’.
3. Once the changes are merged into ‘master’, the issues will be closed.

LRS Testing Group

The LRS Testing Group will continue to meet every 2nd Wednesday of the month to resolve any issues, to discuss the inclusion or removal of any tests, and to discuss upcoming LRS Test Suite updates. The LRS Testing Group will also be leveraged for recommendations to the software update and bug fix processes.

FAQ

In the past, the Test Suite didn't issue certificates, does this one?
Yes it does. The previous versions were beta and release candidate versions. This version is the final version and does issue official certificates.

I passed the Test Suite, why am I not listed in the Conformant LRSs?
You’ll have to self-register and upload your certificate. There is a human-in-the-loop step here to make sure no one asserts on your behalf or uses obviously inaccurate information. Registration in the conformant LRS list is not done automatically as there are use cases where organizations may not want to openly advertise their product.

When will I have to wait for verification/ADL responses?
You’ll have to wait for verification for your LRS Conformance Test Account, xAPI Adopter Account (also used to post Conformant products), and when you’d like a passing Conformance Test to issue a cert.