## 1. Project Overview & Quickstart (nasa/openmct) ## File: README.md # Open MCT [](http://www.apache.org/licenses/LICENSE-2.0) [](https://codecov.io/gh/nasa/openmct) [](https://percy.io/b2e34b17/openmct) [](https://www.npmjs.com/package/openmct) Open MCT (Open Mission Control Technologies) is a next-generation mission control framework for visualization of data on desktop and mobile devices. It is developed at NASA's Ames Research Center, and is being used by NASA for data analysis of spacecraft missions, as well as planning and operation of experimental rover systems. As a generalizable and open source framework, Open MCT could be used as the basis for building applications for planning, operation, and analysis of any systems producing telemetry data. > [!NOTE] > Please visit our [Official Site](https://nasa.github.io/openmct/) and [Getting Started Guide](https://nasa.github.io/openmct/getting-started/) Once you've created something amazing with Open MCT, showcase your work in our GitHub Discussions [Show and Tell](https://github.com/nasa/openmct/discussions/categories/show-and-tell) section. We love seeing unique and wonderful implementations of Open MCT! ## Building and Running Open MCT Locally Building and running Open MCT in your local dev environment is very easy. Be sure you have [Git](https://git-scm.com/downloads) and [Node.js](https://nodejs.org/) installed, then follow the directions below. Need additional information? Check out the [Getting Started](https://nasa.github.io/openmct/getting-started/) page on our website. (These instructions assume you are installing as a non-root user; developers have [reported issues](https://github.com/nasa/openmct/issues/1151) running these steps with root privileges.) 1. Clone the source code: ```sh git clone https://github.com/nasa/openmct.git ``` 2. (Optional) Install the correct node version using [nvm](https://github.com/nvm-sh/nvm): ```sh nvm install ``` 3. Install development dependencies (Note: Check the `package.json` engine for our tested and supported node versions): ```sh npm install ``` 4. Run a local development server: ``` npm start ``` > [!IMPORTANT] > Open MCT is now running, and can be accessed by pointing a web browser at [http://localhost:8080/](http://localhost:8080/) Open MCT is built using [`npm`](http://npmjs.com/) and [`webpack`](https://webpack.js.org/). ## Documentation Documentation is available on the [Open MCT website](https://nasa.github.io/openmct/documentation/). ### Examples The clearest examples for developing Open MCT plugins are in the [tutorials](https://github.com/nasa/openmct-tutorial) provided in our documentation. > [!NOTE] > We want Open MCT to be as easy to use, install, run, and develop for as > possible, and your feedback will help us get there! > Feedback can be provided via [GitHub issues](https://github.com/nasa/openmct/issues/new/choose), > [Starting a GitHub Discussion](https://github.com/nasa/openmct/discussions), > or by emailing us at [arc-dl-openmct@mail.nasa.gov](mailto:arc-dl-openmct@mail.nasa.gov). ## Developing Applications With Open MCT For more on developing with Open MCT, see our documentation for a guide on [Developing Applications with Open MCT](./API.md#starting-an-open-mct-application). ## Compatibility This is a fast moving project and we do our best to test and support the widest possible range of browsers, operating systems, and NodeJS APIs. We have a published list of support available in our package.json's `browserslist` key. The project utilizes `nvm` to maintain consistent node and npm versions across all projects. For UNIX, MacOS, Windows WSL, and other POSIX-compliant shell environments, click [here](https://github.com/nvm-sh/nvm). For Windows, check out [nvm-windows](https://github.com/coreybutler/nvm-windows). If you encounter an issue with a particular browser, OS, or NodeJS API, please [file an issue](https://github.com/nasa/openmct/issues/new/choose). > [!NOTE] > If you are building a project with Open MCT as a git dependency rather than an npm dependency, an extra step is now > necessary to build the source. Due to the threat posed by Supply Chain Attacks `ignore-scripts` is now enabled in > this repository. Developers building with Open MCT as a git dependency will now need to execute `npm run build` > manually. We always encourage developers to use Open MCT as a prebuilt npm repository, rather than building from github. ## Plugins Open MCT can be extended via plugins that make calls to the Open MCT API. A plugin is a group of software components (including source code and resources such as images and HTML templates) that is intended to be added or removed as a single unit. As well as providing an extension mechanism, most of the core Open MCT codebase is also written as plugins. For information on writing plugins, please see [our API documentation](./API.md#plugins). ## Tests Our automated test coverage comes in the form of unit, e2e, visual, performance, and security tests. ### Unit Tests Unit Tests are written for [Jasmine](https://jasmine.github.io/api/edge/global) and run by [Karma](http://karma-runner.github.io). To run: `npm test` The test suite is configured to load any scripts ending with `Spec.js` found in the `src` hierarchy. Full configuration details are found in `karma.conf.js`. By convention, unit test scripts should be located alongside the units that they test; for example, `src/foo/Bar.js` would be tested by `src/foo/BarSpec.js`. ### e2e, Visual, and Performance Testing Our e2e (end-to-end), Visual, and Performance tests leverage the Playwright framework and are executed using Playwright's test runner, [@playwright/test](https://playwright.dev/). #### How to Run Tests - **e2e Tests**: These tests are run on every commit. To run the tests locally, use: ```sh npm run test:e2e:ci ``` - **Visual Tests**: For running the visual test suite, use: ```sh npm run test:e2e:visual ``` - **Performance Tests**: To initiate the performance tests, enter: ```sh npm run test:perf ``` All tests are located within the `e2e/tests/` directory and are identified by the `*.e2e.spec.js` filename pattern. For more information about the e2e test suite, refer to the [README](./e2e/README.md). ### Security Tests Each commit is analyzed for known security vulnerabilities using [CodeQL](https://codeql.github.com/docs/codeql-language-guides/codeql-library-for-javascript/). The list of CWE coverage items is available in the [CodeQL docs](https://codeql.github.com/codeql-query-help/javascript-cwe/). The CodeQL workflow is specified in the [CodeQL analysis file](./.github/workflows/codeql-analysis.yml) and the custom [CodeQL config](./.github/codeql/codeql-config.yml). ### Test Reporting and Code Coverage Each test suite generates a report in CircleCI. For a complete overview of testing functionality, please see our [Circle CI Test Insights Dashboard](https://app.circleci.com/insights/github/nasa/openmct/workflows/the-nightly/overview?branch=master&reporting-window=last-30-days) Our code coverage is generated during the runtime of our unit, e2e, and visual tests. The combination of those reports is published to [codecov.io](https://app.codecov.io/gh/nasa/openmct/) For more on the specifics of our code coverage setup, [see](TESTING.md#code-coverage) ## Glossary Certain terms are used throughout Open MCT with consistent meanings or conventions. Any deviations from the below are issues and should be addressed (either by updating this glossary or changing code to reflect correct usage.) Other developer documentation, particularly in-line documentation, may presume an understanding of these terms. | Term | Definition | |---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | _plugin_ | A removable, reusable grouping of software elements. The application is composed of plugins. | | _composition_ | In the context of a domain object, this term refers to the set of other domain objects that compose or are contained by that object. A domain object's composition is the set of domain objects that should appear immediately beneath it in a tree hierarchy. It is described in its model as an array of ids, providing a means to asynchronously retrieve the actual domain object instances associated with these identifiers. | | _description_ | When used as an object property, this term refers to the human-readable description of a thing, usually a single sentence or short paragraph. It is most often used in the context of extensions, domain object models, or other similar application-specific objects. | | _domain object_ | A meaningful object to the user and a distinct thing in the work supported by Open MCT. Anything that appears in the left-hand tree is a domain object. | | _identifier_ | A tuple consisting of a namespace and a key, which together uniquely identifies a domain object. | | _model_ | The persistent state associated with a domain object. A domain object's model is a JavaScript object that can be converted to JSON without losing information, meaning it contains no methods. | | _name_ | When used as an object property, this term refers to the human-readable name for a thing. It is most often used in the context of extensions, domain object models, or other similar application-specific objects. | | _navigation_ | This term refers to the current state of the application with respect to the user's expressed interest in a specific domain object. For example, when a user clicks on a domain object in the tree, they are navigating to it, and it is thereafter considered the navigated object until the user makes another such choice. | | _namespace_ | A name used to identify a persistence store. A running Open MCT application could potentially use multiple persistence stores. | ## Open MCT v2.0.0 Support for our legacy bundle-based API, and the libraries that it was built on (like Angular 1.x), have now been removed entirely from this repository. For now if you have an Open MCT application that makes use of the legacy API, [a plugin](https://github.com/nasa/openmct-legacy-plugin) is provided that bootstraps the legacy bundling mechanism and API. This plugin will not be maintained over the long term however, and the legacy support plugin will not be tested for compatibility with future versions of Open MCT. It is provided for convenience only. ### How do I know if I am using legacy API? You might still be using legacy API if your source code - Contains files named bundle.js, or bundle.json, - Makes calls to `openmct.$injector()`, or `openmct.$angular`, - Makes calls to `openmct.legacyRegistry`, `openmct.legacyExtension`, or `openmct.legacyBundle`. ### What should I do if I am using legacy API? Please refer to [the modern Open MCT API](https://nasa.github.io/openmct/documentation/). Post any questions to the [Discussions section](https://github.com/nasa/openmct/discussions) of the Open MCT GitHub repository. ## Related Repos > [!NOTE] > Although Open MCT functions as a standalone project, it is primarily an extensible framework intended to be used as a dependency with users' own plugins and packaging. Furthermore, Open MCT is intended to be used with an HTTP server such as Apache or Nginx. A great example of hosting Open MCT with Apache is `openmct-quickstart` and can be found in the table below. | Repository | Description | | --- | --- | | [openmct-tutorial](https://github.com/nasa/openmct-tutorial) | A great place for beginners to learn how to use and extend Open MCT. | | [openmct-quickstart](https://github.com/scottbell/openmct-quickstart) | A working example of Open MCT integrated with Apache HTTP server, YAMCS telemetry, and Couch DB for persistence. | [Open MCT YAMCS Plugin](https://github.com/akhenry/openmct-yamcs) | Plugin for integrating YAMCS telemetry and command server with Open MCT. | | [openmct-performance](https://github.com/unlikelyzero/openmct-performance) | Resources for performance testing Open MCT. | | [openmct-as-a-dependency](https://github.com/unlikelyzero/openmct-as-a-dependency) | An advanced guide for users on how to build, develop, and test Open MCT when it's used as a dependency. | --- ## File: docs/src/guide/security.md # Security Guide Open MCT is a rich client with plugin support that executes as a single page web application in a browser environment. Security concerns and vulnerabilities associated with the web as a platform should be considered before deploying Open MCT (or any other web application) for mission or production usage. This document describes several important points to consider when developing for or deploying Open MCT securely. Other resources such as [Open Web Application Security Project (OWASP)](https://www.owasp.org) provide a deeper and more general overview of security for web applications. ## Security Model Open MCT has been architected assuming the following deployment pattern: * A tagged, tested Open MCT version will be used. * Externally authored plugins will be installed. * A server will provide persistent storage, telemetry, and other shared data. * Authorization, authentication, and auditing will be handled by a server. ## Security Procedures The Open MCT team secures our code base using a combination of code review, dependency review, and periodic security reviews. Static analysis performed during automated verification additionally safeguards against common coding errors which may result in vulnerabilities. ### Code Review All contributions are reviewed by internal team members. External contributors receive increased scrutiny for security and quality, and must sign a licensing agreement. ### Dependency Review Before integrating third-party dependencies, they are reviewed for security and quality, with consideration given to authors and users of these dependencies, as well as review of open source code. ### Periodic Security Reviews Open MCT's code, design, and architecture are periodically reviewed (approximately annually) for common security issues, such as the [OWASP Top Ten](https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project). ## Security Concerns Certain security concerns deserve special attention when deploying Open MCT, or when authoring plugins. ### Identity Spoofing Open MCT issues calls to web services with the privileges of a logged in user. Compromised sources (either for Open MCT itself or a plugin) could therefore allow malicious code to execute with those privileges. To avoid this: * Serve Open MCT and other scripts over SSL (https rather than http) to prevent man-in-the-middle attacks. * Exercise precautions such as security reviews for any plugins or applications built for or with Open MCT to reject malicious changes. ### Information Disclosure If Open MCT is used to handle or display sensitive data, any components (such as adapter plugins) must take care to avoid leaking or disclosing this information. For example, avoid sending sensitive data to third-party servers or insecure APIs. ### Data Tampering The web application architecture leaves open the possibility that direct calls will be made to back-end services, circumventing Open MCT entirely. As such, Open MCT assumes that server components will perform any necessary data validation during calls issues to the server. Additionally, plugins which serialize and write data to the server must escape that data to avoid database injection attacks, and similar. ### Repudiation Open MCT assumes that servers log any relevant interactions and associates these with a user identity; the specific user actions taken within the application are assumed not to be of concern for auditing. In the absence of server-side logging, users may disclaim (maliciously, mistakenly, or otherwise) actions taken within the system without any way to prove otherwise. If keeping client-level interactions is important, this will need to be implemented via a plugin. ### Denial-of-service Open MCT assumes that server-side components will be insulated against denial-of-service attacks. Services should only permit resource-intensive tasks to be initiated by known or trusted users. ### Elevation of Privilege Corollary to the assumption that servers guide against identity spoofing, Open MCT assumes that services do not allow a user to act with inappropriately escalated privileges. Open MCT cannot protect against such escalation; in the clearest case, a malicious actor could interact with web services directly to exploit such a vulnerability. ## Additional Reading The following resources have been used as a basis for identifying potential security threats to Open MCT deployments in preparation of this document: * [STRIDE model](https://www.owasp.org/index.php/Threat_Risk_Modeling#STRIDE) * [Attack Surface Analysis Cheat Sheet](https://www.owasp.org/index.php/Attack_Surface_Analysis_Cheat_Sheet) * [XSS Prevention Cheat Sheet](https://www.owasp.org/index.php/XSS_(Cross_Site_Scripting)_Prevention_Cheat_Sheet) --- ## File: docs/src/process/testing/plan.md # Test Plan ## Test Levels Testing for Open MCT includes: * _Smoke testing_: Brief, informal testing to verify that no major issues or regressions are present in the software, or in specific features of the software. * _Unit testing_: Automated verification of the performance of individual software components. * _User testing_: Testing with a representative user base to verify that application behaves usably and as specified. * _Long-duration testing_: Testing which takes place over a long period of time to detect issues which are not readily noticeable during shorter test periods. ### Smoke Testing Manual, non-rigorous testing of the software and/or specific features of interest. Verifies that the software runs and that basic functionality is present. The outcome of Smoke Testing should be a simplified list of Acceptance Tests which could be executed by another team member with sufficient context. ### Unit Testing Unit tests are automated tests which exercise individual software components. Tests are subject to code review along with the actual implementation, to ensure that tests are applicable and useful. Unit tests should meet [test standards](https://github.com/nasa/openmctweb/blob/master/CONTRIBUTING.md#test-standards) as described in the contributing guide. ### User Testing User testing is performed at scheduled times involving target users of the software or reasonable representatives, along with members of the development team exercising known use cases. Users test the software directly; the software should be configured as similarly to its planned production configuration as is feasible without introducing other risks (e.g. damage to data in a production instance.) User testing will focus on the following activities: * Verifying issues resolved since the last test session. * Checking for regressions in areas related to recent changes. * Using major or important features of the software, as determined by the user. * General "trying to break things." During user testing, users will [report issues](https://github.com/nasa/openmct/issues/new/choose) as they are encountered. Desired outcomes of user testing are: * Identified software defects. * Areas for usability improvement. * Feature requests (particularly missed requirements.) * Recorded issue verification. ### Long-duration Testing Long-duration testing occurs over a twenty-four hour period. The software is run in one or more stressing cases representative of expected usage. After twenty-four hours, the software is evaluated for: * Performance metrics: Have memory usage or CPU utilization increased during this time period in unexpected or undesirable ways? * Subjective usability: Does the software behave in the same way it did at the start of the test? Is it as responsive? Any defects or unexpected behavior identified during testing should be [reported as issues](https://github.com/nasa/openmct/issues/new/choose) and reviewed for severity. ## Test Performance Tests are performed at various levels of frequency. * _Per-merge_: Performed before any new changes are integrated into the software. * _Per-sprint_: Performed at the end of every [sprint](../cycle.md). * _Per-release_: Performed at the end of every [release](../cycle.md). ### Per-merge Testing Before changes are merged, the author of the changes must perform: * _Smoke testing_ (both generally, and for areas which interact with the new changes.) * _Unit testing_ (as part of the automated build step.) Changes are not merged until the author has affirmed that both forms of testing have been performed successfully; this is documented by the [Author Checklist](https://github.com/nasa/openmctweb/blob/master/CONTRIBUTING.md#author-checklist). ### Per-sprint Testing Before a sprint is closed, the development team must additionally perform: * A relevant subset of [_user testing_](procedures.md#user-test-procedures) identified by the acting [project manager](../cycle.md#roles). * [_Long-duration testing_](procedures.md#long-duration-testing) (specifically, for 24 hours.) Issues are reported as a product of both forms of testing. A sprint is not closed until both categories have been performed on the latest snapshot of the software, _and_ no issues labelled as ["blocker"](https://github.com/nasa/openmctweb/blob/master/CONTRIBUTING.md#issue-reporting) remain open. ### Per-release Testing As [per-sprint testing](#per-sprint-testing), except that _user testing_ should cover all test cases, with less focus on changes from the specific sprint or release. Per-release testing should also include any acceptance testing steps agreed upon with recipients of the software. A release is not closed until both categories have been performed on the latest snapshot of the software, _and_ no issues labelled as ["blocker" or "critical"](https://github.com/nasa/openmctweb/blob/master/CONTRIBUTING.md#issue-reporting) remain open. ### Testathons Testathons can be used as a means of performing per-sprint and per-release testing. #### Timing For per-sprint testing, a testathon is typically performed at the beginning of the third week of a sprint, and again later that week to verify any fixes. For per-release testing, a testathon is typically performed prior to any formal testing processes that are applicable to that release. #### Process 1. Prior to the scheduled testathon, a list will be compiled of all issues that are closed and unverified. 2. For each issue, testers should review the associated PR for testing instructions. See the contributing guide for instructions on [pull requests](https://github.com/nasa/openmct/blob/master/CONTRIBUTING.md#merging). 3. As each issue is verified via testing, any team members testing it should leave a comment on that issue indicating that it has been verified fixed. 4. If a bug is found that relates to an issue being tested, notes should be included on the associated issue, and the issue should be reopened. Bug notes should include reproduction steps. 5. For any bugs that are not obviously related to any of the issues under test, a new issue should be created with details about the bug, including reproduction steps. If unsure about whether a bug relates to an issue being tested, just create a new issue. 6. At the end of the testathon, triage will take place, where all tested issues will be reviewed. 7. If verified fixed, an issue will remain closed, and will have the “unverified” label removed. 8. For any bugs found, a severity will be assigned. 9. A second testathon will be scheduled for later in the week that will aim to address all issues identified as blockers, as well as any other issues scoped by the team during triage. 10. Any issues that were not tested will remain "unverified" and will be picked up in the next testathon. --- ## File: docs/src/process/cycle.md # Development Cycle Development of Open MCT occurs on an iterative cycle of sprints and releases. * A _sprint_ is three weeks in duration, and represents a set of improvements that can be completed and tested by the development team. Software at the end of the sprint is "semi-stable"; it will have undergone reduced testing and may carry defects or usability issues of lower severity, particularly if there are workarounds. * A _release_ occurs every four sprints. Releases are stable, and will have undergone full acceptance testing to ensure that the software behaves correctly and usably. ## Roles The sprint process assumes the presence of a __project manager.__ The project manager is responsible for making tactical decisions about what development work will be performed, and for coordinating with stakeholders to arrive at higher-level strategic decisions about desired functionality and characteristics of the software, major external milestones, and so forth. In the absence of a dedicated project manager, this role may be rotated among members of the development team on a per-sprint basis. Responsibilities of the project manager including: * Maintaining (with agreement of stakeholders) a "road map" of work planned for future releases/sprints; this should be higher-level, usually expressed as "themes", with just enough specificity to gauge feasibility of plans, relate work back to milestones, and identify longer-term dependencies. * Determining (with assistance from the rest of the team) which issues to work on in a given sprint and how they shall be assigned. * Pre-planning subsequent sprints to ensure that all members of the team always have a clear direction. * Scheduling and/or ensuring adherence to [process points](#process-points). * Responding to changes within the sprint (shifting priorities, new issues) and re-allocating work for the sprint as needed. ## Sprint Calendar Certain [process points](#process-points) are regularly scheduled in the sprint cycle. ### Sprints by Release Allocation of work among sprints should be planned relative to release goals and milestones. As a general guideline, higher-risk work (large new features which may carry new defects, major refactoring, design changes with uncertain effects on usability) should be allocated to earlier sprints, allowing for time in later sprints to ensure stability. | Sprint | Focus | |:------:|:--------------------------------------------------------| | __1__ | Prototyping, design, experimentation. | | __2__ | New features, refinements, enhancements. | | __3__ | Feature completion, low-risk enhancements, bug fixing. | | __4__ | Stability & quality assurance. | ### Sprints 1-3 The first three sprints of a release are primarily centered around development work, with regular acceptance testing in the third week. During this third week, the top priority should be passing acceptance testing (e.g. by resolving any blockers found); any resources not needed for this effort should be used to begin work for the subsequent sprint. | Week | Mon | Tue | Wed | Thu | Fri | |:-----:|:-------------------------:|:------:|:---:|:----------------------------:|:-------------------------------------:| | __1__ | Sprint plan | Tag-up | | | | | __2__ | | Tag-up | | | Code freeze and sprint branch | | __3__ | Per-sprint testing | Triage | | _Per-sprint testing*_ | Ship and merge sprint branch to master| * If necessary. ### Sprint 4 The software must be stable at the end of the fourth sprint; because of this, the fourth sprint is scheduled differently, with a heightened emphasis on testing. | Week | Mon | Tue | Wed | Thu | Fri | |-------:|:-------------------------:|:------:|:---:|:----------------------------:|:-----------:| | __1__ | Sprint plan | Tag-up | | | Code freeze | | __2__ | Per-release testing | Triage | | | | | __3__ | _Per-release testing*_ | Triage | | _Per-release testing*_ | Ship | * If necessary. ## Process Points * __Sprint plan.__ Project manager allocates issues based on theme(s) for sprint, then reviews with team. Each team member should have roughly two weeks of work allocated (to allow time in the third week for testing of work completed.) * Project manager should also sketch out subsequent sprint so that team may begin work for that sprint during the third week, since testing and blocker resolution is unlikely to require all available resources. * Testing success criteria identified per issue (where necessary). This could be in the form of acceptance tests on the issue or detailing performance tests, for example. * __Tag-up.__ Check in and status update among development team. May amend plan for sprint as-needed. * __Code freeze.__ Any new work from this sprint (features, bug fixes, enhancements) must be integrated by the end of the second week of the sprint. After code freeze, a sprint branch will be created (and until the end of the sprint) the only changes that should be merged into the sprint branch should directly address issues needed to pass acceptance testing. During this time, any other feature development will continue to be merged into the master branch for the next sprint. * __Sprint branch merge to master.__ After acceptance testing, the sprint branch will be merged back to the master branch. Any code conflicts that arise will be resolved by the team. * [__Per-release Testing.__](testing/plan.md#per-release-testing) Structured testing with predefined success criteria. No release should ship without passing acceptance tests. Time is allocated in each sprint for subsequent rounds of acceptance testing if issues are identified during a prior round. Specific details of acceptance testing need to be agreed-upon with relevant stakeholders and delivery recipients, and should be flexible enough to allow changes to plans (e.g. deferring delivery of some feature in order to ensure stability of other features.) Baseline testing includes: * [__Testathon.__](testing/plan.md#user-testing) Multi-user testing, involving as many users as is feasible, plus development team. Open-ended; should verify completed work from this sprint using the sprint branch, test exploratory for regressions, et cetera. * [__Long-Duration Test.__](testing/plan.md#long-duration-testing) A test to verify that the software remains stable after running for longer durations. May include some combination of automated testing and user verification (e.g. checking to verify that software remains subjectively responsive at conclusion of test.) * [__Unit Testing.__](testing/plan.md#unit-testing) Automated testing integrated into the build. (These tests are verified to pass more often than once per sprint, as they run before any merge to master, but still play an important role in per-release testing.) * [__Per-sprint Testing.__](testing/plan.md#per-sprint-testing) Subset of Pre-release Testing which should be performed before shipping at the end of any sprint. Time is allocated for a second round of Pre-release Testing if the first round is not passed. Smoke tests collected from issues/PRs * __Triage.__ Team reviews issues from acceptance testing and uses success criteria to determine whether or not they should block release, then formulates a plan to address these issues before the next round of acceptance testing. Focus here should be on ensuring software passes that testing in order to ship on time; may prefer to disable malfunctioning components and fix them in a subsequent sprint, for example. * [__Ship.__](version.md) Tag a code snapshot that has passed release/sprint testing and deploy that version. (Only true if relevant testing has passed by this point; if testing has not been passed, will need to make ad hoc decisions with stakeholders, e.g. "extend the sprint" or "defer shipment until end of next sprint.") --- ## File: docs/src/process/index.md # Development Process The process used to develop Open MCT is described in the following documents: * The [Development Cycle](cycle.md) describes how and when specific process points are repeated during development. * The [Version Guide](version.md) describes version numbering for Open MCT (both semantics and process.) * The [Test Plan](testing/plan.md) summarizes the approaches used to test Open MCT. --- ## File: docs/src/process/release.md # Release of NASA Open MCT NPM Package This document outlines the process and key considerations for releasing a new version of the NASA Open MCT project as an NPM (Node Package Manager) package. ## 1. Pre-requisites Before releasing a new version of the NASA Open MCT NPM package, ensure all dependencies are updated, and comprehensive tests are performed. This ensures compatibility and performance of the Open MCT within the Node.js ecosystem. ## 2. Versioning Versioning is a critical step for package release. The Open MCT team follows [Semantic Versioning (SemVer)](https://semver.org) that consists of three major components: MAJOR.MINOR.PATCH. These ensure a structured process for updating, bug fixes, backward compatibility, and software progress. ## 3. Changelog Maintenance A comprehensive changelog file, `CHANGELOG.md`, documents any changes, adding a high level of transparencies for anyone desiring to look into the status of new and past progress. It includes the summation of any major new enhancements, changes, bug fixes, and the credits to the users responsible for each unique progress. ## 4. Notable Changes Labels on GitHub PRs For the Open MCT package, we leverage GitHub's Pull Request (PR) mechanisms extensively, with three important PR labels dedicated to signifying 'notable_changes': - **Breaking Change** Highlights the integration of changes that are suspected to break, or without a doubt will break, backward compatibility. These should signal to users the upgrade might be seamless only if dependency and integration factors are properly managed, if not, one should expect to manage atypical technical snags. - **API change** Signifies when a contribution makes any complete or under layer changes to the communication or its supporting access processes. This label flags required see-through insight on how the web-based control panel sees and manipulates any value and or network logs. - **Default Behavior Change:** In the incident an update either adjusts a form to or integrates a not previously kept setting or plugin. i.e. autoscale is enabled by default when working with plots. ## 6. Community & Contributions A flat community and the rounded center are kept in continuous celebration, with the given station open for two open-specifying dialogues, research, and all-for development probing. State the ownership for a handed looped, a welcome for even structure-core and architectural draft and impend. Thank you for your collaboration and commitment to moving the project onto a text big club. --- ## File: docs/src/process/version.md # Version Guide This document describes semantics and processes for providing version numbers for Open MCT, and additionally provides guidelines for dependent projects developed by the same team. Versions are incremented at specific points in Open MCT's [Development Cycle](cycle.md); see that document for a description of sprints and releases. ## Audience Individuals interested in consuming version numbers can be categorized as follows: * _Users_: Generally disinterested, occasionally wish to identify version to cross-reference against documentation, or to report issues. * _Testers_: Want to identify which version of the software they are testing, e.g. to file issues for defects. * _Internal developers_: Often, inverse of testers; want to identify which version of software was/is in use when certain behavior is observed. Want to be able to correlate versions in use with “streams” of development (e.g. dev vs. prod), when possible. * _External developers_: Need to understand which version of software is in use when developing/maintaining plug-ins, in order to ensure compatibility of their software. ## Version Reporting Software versions should be reflected in the user interface of the application in three ways: * _Version number_: A semantic version (see below) which serves both to uniquely identify releases, as well as to inform plug-in developers about compatibility with previous releases. * _Revision identifier_: While using git, the commit hash. Supports internal developers and testers by uniquely identifying client software snapshots. * _Branding_: Identifies which variant is in use. (Typically, Open MCT is re-branded when deployed for a specific mission or center.) ## Version Numbering Open MCT shall provide version numbers consistent with [Semantic Versioning 2.0.0](http://semver.org/). In summary, versions are expressed in a "major.minor.patch" form, and incremented based on nature of changes to external API. Breaking changes require a "major" version increment; backwards-compatible changes require a "minor" version increment; neutral changes (such as bug fixes) require a "patch" version increment. A hyphen-separated suffix indicates a pre-release version, which may be unstable or may not fully meet compatibility requirements. Additionally, the following project-specific standards will be used: * During development, a "-next" suffix shall be appended to the version number. The version number before the suffix shall reflect the next expected version number for release. * Prior to a 1.0.0 release, the _minor_ version will be incremented on a per-release basis; the _patch_ version will be incremented on a per-sprint basis. * Starting at version 1.0.0, version numbers will be updated with each completed sprint. The version number for the sprint shall be determined relative to the previous released version; the decision to increment the _major_, _minor_, or _patch_ version should be made based on the nature of changes during that release. (It is recommended that these numbers are incremented as changes are introduced, such that at end of release the version number may be chosen by simply removing the suffix.) * The first three sprints in a release may be unstable; in these cases, a unique version identifier should still be generated, but a suffix should be included to indicate that the version is not necessarily production-ready. Recommended suffixes are: Sprint | Suffix :------:|:--------: 1 | `-alpha` 2 | `-beta` 3 | `-rc` ### Scope of External API "External API" refers to the API exposed to, documented for, and used by plug-in developers. Changes to interfaces used internally by Open MCT (or otherwise not documented for use externally) require only a _patch_ version bump. ## Incrementing Versions At the end of a sprint, the [project manager](cycle.md#roles) should update (or delegate the task of updating) Open MCT version numbers by the following process: 1. Update version number in `package.json` 1. Checkout branch created for the last sprint that has been successfully tested. 2. Remove a `-next` suffix from the version in `package.json`. 3. Verify that resulting version number meets semantic versioning requirements relative to previous stable version. Increment the version number if necessary. 4. If version is considered unstable (which may be the case during the first three sprints of a release), apply a new suffix per [Version Numbering](#version-numbering) guidance above. 2. Tag the release. 1. Commit changes to `package.json` on the new branch created in the previous step. The commit message should reference the sprint being closed, preferably by a URL reference to the associated Milestone in GitHub. 2. Verify that build still completes, that application passes smoke-testing, and that only differences from tested versions are the changes to version number above. 3. Push the new branch. 4. Tag this commit with the version number, prepending the letter "v". (e.g. `git tag v0.9.3-alpha`) 5. Push the tag to GitHub. (e.g. `git push origin v0.9.3-alpha`). 3. Upload a release archive. 1. Use the [GitHub release interface](https://github.com/nasa/openmct/releases) to draft a new release. 2. Choose the existing tag for the new version (created and pushed above.) Enter the tag name as the release name as well; see existing releases for examples. (e.g. `Open MCT v0.9.3-alpha`) 3. Designate the release as a "pre-release" as appropriate (for instance, when the version number has been suffixed as unstable, or when the version number is below 1.0.0.) 4. Add release notes including any breaking changes, enhancements, bug fixes with solutions in brief. 5. Publish the release. 4. Publish the release to npm 1. Login to npm 2. Checkout the tag created in the previous step. 3. In `package.json` change package to be public (private: false) 4. Test the package before publishing by doing `npm publish --dry-run` if necessary. 5. Publish the package to the npmjs registry (e.g. `npm publish --access public`) NOTE: Use the `--tag unstable` flag to the npm publish if this is a prerelease. 6. Confirm the package has been published (e.g. `https://www.npmjs.com/package/openmct`) 5. Update snapshot status in `package.json` 1. Create a new branch off the `master` branch. 2. Remove any suffix from the version number, or increment the _patch_ version if there is no suffix. 3. Append a `-next` suffix. 4. Commit changes to `package.json` on the `master` branch. The commit message should reference the sprint being opened, preferably by a URL reference to the associated Milestone in GitHub. 5. Verify that build still completes, that application passes smoke-testing. 6. Create a PR to be merged into the `master` branch. Projects dependent on Open MCT being co-developed by the Open MCT team should follow a similar process, except that they should additionally update their dependency on Open MCT to point to the latest archive when removing their `-next` status, and that they should be pointed back to the `master` branch after this has completed. --- ## File: docs/src/index.md # Open MCT Documentation ## Overview Documentation is provided to support the use and development of Open MCT. It's recommended that before doing any development with Open MCT you take some time to familiarize yourself with the documentation below. Open MCT provides functionality out of the box, but it's also a platform for building rich mission operations applications based on modern web technology. The platform is configured by plugins which extend the platform at a variety of extension points. The details of how to extend the platform are provided in the following documentation. ## Sections * The [API](api/) document is generated from inline documentation using [JSDoc](http://usejsdoc.org/), and describes the JavaScript objects and functions that make up the software platform. * The [Development Process](process/) document describes the Open MCT software development cycle. * The [Tutorials](https://github.com/nasa/openmct-tutorial) give examples of extending the platform to add functionality, and integrate with data sources. --- ## File: e2e/README.md # e2e testing This document captures information specific to the e2e testing of Open MCT. For general information about testing, please see [the Open MCT README](https://github.com/nasa/openmct/blob/master/README.md#tests). ## Table of Contents This document is designed to capture on the What, Why, and How's of writing and running e2e tests in Open MCT. Please use the built-in Github Table of Contents functionality at the top left of this page or the markup. 1. [Getting Started](#getting-started) 2. [Types of Testing](#types-of-e2e-testing) 3. [Architecture](#test-architecture-and-ci) ## Getting Started While our team does our best to lower the barrier to entry to working with our e2e framework and Open MCT, there is a bit of work required to get from 0 to 1 test contributed. ### Getting started with Playwright If this is your first time ever using the Playwright framework, we recommend going through the [Getting Started Guide](https://playwright.dev/docs/next/intro) which can be completed in about 15 minutes. This will give you a concise tour of Playwright's functionality and an understanding of the official Playwright documentation which we leverage in Open MCT. ### Getting started with Open MCT's implementation of Playwright Once you've got an understanding of Playwright, you'll need a baseline understanding of Open MCT: 1. Follow the steps [Building and Running Open MCT Locally](../README.md#building-and-running-open-mct-locally) 2. Once you're serving Open MCT locally, create a 'Display Layout' object. Save it. 3. Create a 'Plot' Object (e.g.: 'Stacked Plot') 4. Create an Example Telemetry Object (e.g.: 'Sine Wave Generator') 5. Expand the Tree and note the hierarchy of objects which were created. 6. Navigate to the Demo Display Layout Object to edit and modify the embedded plot. 7. Modify the embedded plot with Telemetry Data. What you've created is a display which mimics the display that a mission control operator might use to understand and model telemetry data. Recreate the steps above with Playwright's codegen tool: 1. `npm run start` in a terminal window to serve Open MCT locally 2. `npx @playwright/test install` to install playwright and dependencies 3. Open another terminal window and start the Playwright codegen application `npx playwright codegen` 4. Navigate the browser to `http://localhost:8080` 5. Click the Create button and notice how your actions in the browser are being recorded in the Playwright Inspector 6. Continue through the steps 2-6 above What you've created is an automated test which mimics the creation of a mission control display. Next, you should walk through our implementation of Playwright in Open MCT: 1. Close any terminals which are serving up a local instance of Open MCT 2. Run our 'Getting Started' test in debug mode with `npm run test:e2e:local -- exampleTemplate --debug` 3. Step through each test step in the Playwright Inspector to see how we leverage Playwright's capabilities to test Open MCT ## Types of e2e Testing e2e testing describes the layer at which a test is performed without prescribing the assertions which are made. Generally, when writing an e2e test, we have five choices to make on an assertion strategy: 1. Functional - Verifies the functional correctness of the application. Sometimes interchanged with e2e or regression testing. 2. Visual - Verifies the "look and feel" of the application and can only detect _undesirable changes when compared to a previous baseline_. 3. Snapshot - Similar to Visual in that it captures the "look" of the application and can only detect _undesirable changes when compared to a previous baseline_. **Generally not preferred due to advanced setup necessary.** 4. Accessibility - Verifies that the application meets the accessibility standards defined by the [WCAG organization](https://www.w3.org/WAI/standards-guidelines/wcag/). 5. Performance - Verifies that application provides a performant experience. Like Snapshot testing, these tests are generally not recommended due to their difficulty in providing a consistent result. When choosing between the different testing strategies, think only about the assertion that is made at the end of the series of test steps. "I want to verify that the Timer plugin functions correctly" vs "I want to verify that the Timer plugin does not look different than originally designed". We do not want to interleave visual and functional testing inside the same suite because visual test verification of correctness must happen with a 3rd party service. This service is not available when executing these tests in other contexts (i.e. VIPER). ### Functional Testing The bulk of our e2e coverage lies in "functional" test coverage which verifies that Open MCT is functionally correct as well as defining _how we expect it to behave_. This enables us to test the application exactly as a user would, while prescribing exactly how a user can interact with the application via a web browser. ### Visual Testing Visual Testing is an essential part of our e2e strategy as it ensures that the application _appears_ correctly to a user while it compliments the functional e2e suite. It would be impractical to make thousands of assertions functional assertions on the look and feel of the application. Visual testing is interested in getting the DOM into a specified state and then comparing that it has not changed against a baseline. For a better understanding of the visual issues which affect Open MCT, please see our bug tracker with the `label:visual` filter applied [here](https://github.com/nasa/openmct/issues?q=label%3Abug%3Avisual+) To read about how to write a good visual test, please see [How to write a great Visual Test](#how-to-write-a-great-visual-test). `npm run test:e2e:visual` commands will run all of the visual tests against a local instance of Open MCT. If no `PERCY_TOKEN` API key is found in the terminal or command line environment variables, no visual comparisons will be made. - `npm run test:e2e:visual:ci` will run against every commit and PR. - `npm run test:e2e:visual:full` will run every night with additional comparisons made for Larger Displays and with the `snow` theme. #### Percy.io To make this possible, we're leveraging a 3rd party service, [Percy](https://percy.io/). This service maintains a copy of all changes, users, scm-metadata, and baselines to verify that the application looks and feels the same _unless approved by a Open MCT developer_. To request a Percy API token, please reach out to the Open MCT Dev team on GitHub. For more information, please see the official [Percy documentation](https://docs.percy.io/docs/visual-testing-basics). At present, we are using percy with two configuration files: `./e2e/.percy.nightly.yml` and `./e2e/.percy.ci.yml`. This is mainly to reduce the number of snapshots. ### Advanced: Snapshot Testing (Not Recommended) While snapshot testing offers a precise way to detect changes in your application without relying on third-party services like Percy.io, we've found that it doesn't offer any advantages over visual testing in our use-cases. Therefore, snapshot testing is **not recommended** for further implementation. #### CI vs Manual Checks Snapshot tests can be reliably executed in Continuous Integration (CI) environments but lack the manual oversight provided by visual testing platforms like Percy.io. This means they may miss issues that a human reviewer could catch during manual checks. #### Example A single visual test assertion in Percy.io can be executed across 10 different browser and resolution combinations without additional setup, providing comprehensive testing with minimal configuration. In contrast, a snapshot test is restricted to a single OS and browser resolution, requiring more effort to achieve the same level of coverage. #### Further Reading For those interested in the mechanics of snapshot testing with Playwright, you can refer to the [Playwright Snapshots Documentation](https://playwright.dev/docs/test-snapshots). However, keep in mind that we do not recommend using this approach. #### Open MCT's implementation - Our Snapshot tests receive a `@snapshot` tag. - Snapshots need to be executed within the official Playwright container to ensure we're using the exact rendering platform in CI and locally. To do a valid comparison locally: - Note, microsoft might have retired older images like `-jammy`. Run the following command to find the available tags: `curl -s https://mcr.microsoft.com/v2/playwright/tags/list | jq -r '.tags[]' | grep "v{X.X.X}` ```sh // Replace {X.X.X} with the current Playwright version // from our package.json configuration file docker run --rm --network host -v $(pwd):/work/ -w /work/ -it mcr.microsoft.com/playwright:v{X.X.X}-jammy /bin/bash npm install npm run test:e2e:checksnapshots ``` ### Updating Snapshots When the `@snapshot` tests fail, they will need to be evaluated to determine if the failure is an acceptable and desirable or an unintended regression. To compare a snapshot, run a test and open the html report with the 'Expected' vs 'Actual' screenshot. If the actual screenshot is preferred, then the source-controlled 'Expected' snapshots will need to be updated with the following scripts. ```sh // Replace {X.X.X} with the current Playwright version // from our package.json configuration file docker run --rm --network host -v $(pwd):/work/ -w /work/ -it mcr.microsoft.com/playwright:v{X.X.X}-jammy /bin/bash npm install npm run test:e2e:updatesnapshots ``` Once that's done, you'll need to run the following to verify that the changes do not cause more problems: ```sh npm run test:e2e:checksnapshots ``` ## Automated Accessibility (a11y) Testing Open MCT incorporates accessibility testing through two primary methods to ensure its compliance with accessibility standards: 1. **Usage of Playwright's Locator Strategy**: Open MCT utilizes Playwright's locator strategy, specifically the [page.getByRole('') function](https://playwright.dev/docs/api/class-framelocator#frame-locator-get-by-role), to ensure that web elements are accessible via assistive technologies. This approach focuses on the accessibility of elements rather than full adherence to a11y guidelines, which is covered in the second method. 2. **Enforcing a11y Guidelines with Playwright Axe Plugin**: To rigorously enforce a11y guideline compliance, Open MCT employs the [playwright axe plugin](https://playwright.dev/docs/accessibility-testing). This is achieved through the `scanForA11yViolations` function within the visual testing suite. This method not only benefits from the existing coverage of the visual tests but also targets specific a11y issues, such as `color-contrast` violations, which are particularly pertinent in the context of visual testing. ### a11y Standards (WCAG and Section 508) Playwright axe supports a wide range of [WCAG Standards](https://playwright.dev/docs/accessibility-testing#scanning-for-wcag-violations) to test against. Open MCT is testing against the [Section 508](https://www.section508.gov/test/testing-overview/) accessibility guidelines with the intent to support higher standards over time. As of 2024, Section508 requirements now map completely to WCAG 2.0 AA. In the future, Section 508 requirements may map to WCAG 2.1 AA. ### Reading an a11y test failure When an a11y test fails, the result must be interpreted in the html test report or the a11y report json artifact stored in the `/test-results/` folder. The json structure should be parsed for `"violations"` by `"id"` and identified `"target"`. Example provided for the 'color-contrast-enhanced' violation. ```json "violations": { "id": "color-contrast-enhanced", "impact": "serious", "html": "0 Snapshots Show", "target": [ ".s-status-off > .label.c-indicator__label" ], "failureSummary": "Fix any of the following:\n Element has insufficient color contrast of 6.51 (foreground color: #aaaaaa, background color: #262626, font size: 8.1pt (10.8px), font weight: normal). Expected contrast ratio of 7:1" } ``` ## Performance Testing The open source performance tests function in three ways which match their naming and folder structure: `tests/performance` - The tests at the root of this folder path detect functional changes which are mostly apparent with large performance regressions like [this](https://github.com/nasa/openmct/issues/6879). These tests run against openmct webpack in `production-mode` with the `npm run test:perf:localhost` script. `tests/performance/contract/` - These tests serve as [contracts](https://martinfowler.com/bliki/ContractTest.html) for the locator logic, functionality, and assumptions will work in our downstream, closed source test suites. These tests run against openmct webpack in `dev-mode` with the `npm run test:perf:contract` script. `tests/performance/memory/` - These tests execute memory leak detection checks in various ways. This is expected to evolve as we move to the `memlab` project. These tests run against openmct webpack in `production-mode` with the `npm run test:perf:memory` script. These tests are expected to become blocking and gating with assertions as we extend the capabilities of Playwright. In addition to the explicit definition of performance tests, we also ensure that our test timeout timing is "tight" to catch performance regressions detectable by action timeouts. i.e. [Notebooks load much slower than they used to #6459](https://github.com/nasa/openmct/issues/6459) ## Test Architecture and CI ### Architecture ### File Structure Our file structure follows the type of type of testing being exercised at the e2e layer and files containing test suites which matcher application behavior or our `src` and `example` layout. This area is not well refined as we figure out what works best for closed source and downstream projects. This may change altogether if we move `e2e` to it's own npm package. |File Path|Description| |:-:|-| |`./helper` | Contains helper functions or scripts which are leveraged directly within the test suites (e.g.: non-default plugin scripts injected into the DOM)| |`./test-data` | Contains test data which is leveraged or generated in the functional, performance, or visual test suites (e.g.: localStorage data).| |`./tests/functional` | The bulk of the tests are contained within this folder to verify the functionality of Open MCT.| |`./tests/functional/example/` | Tests which specifically verify the example plugins (e.g.: Sine Wave Generator).| |`./tests/functional/plugins/` | Tests which loosely test each plugin. This folder is the most likely to change. Note: some `@snapshot` tests are still contained within this structure.| |`./tests/framework/` | Tests which verify that our testing framework's functionality and assumptions will continue to work based on further refactoring or Playwright version changes (e.g.: verifying custom fixtures and appActions).| |`./tests/performance/` | Performance tests which should be run on every commit.| |`./tests/performance/contract/` | A subset of performance tests which are designed to provide a contract between the open source tests which are run on every commit and the downstream tests which are run post merge and with other frameworks.| |`./tests/performance/memory` | A subset of performance tests which are designed to test for memory leaks.| |`./tests/visual-a11y/` | Visual tests and accessibility tests.| |`./tests/visual-a11y/component/` | Visual and accessibility tests which are only run against a single component.| |`./appActions.js` | Contains common methods which can be leveraged by test case authors to quickly move through the application when writing new tests.| |`./baseFixture.js` | Contains base fixtures which only extend default `@playwright/test` functionality. The expectation is that these fixtures will be removed as the native Playwright API improves| Our functional tests end in `*.e2e.spec.js`, visual tests in `*.visual.spec.js` and performance tests in `*.perf.spec.js`. ### Configuration Where possible, we try to run Open MCT without modification or configuration change so that the Open MCT doesn't fail exclusively in "test mode" or in "production mode". Open MCT is leveraging the [config file](https://playwright.dev/docs/test-configuration) pattern to describe the capabilities of Open MCT e2e _where_ it's run |Config File|Description| |:-:|-| |`./playwright-ci.config.js` | Used when running in CI or to debug CI issues locally| |`./playwright-local.config.js` | Used when running locally| |`./playwright-performance.config.js` | Used when running performance tests in CI or locally| |`./playwright-performance-devmode.config.js` | Used when running performance tests in CI or locally| |`./playwright-visual-a11y.config.js` | Used to run the visual and a11y tests in CI or locally| #### Test Tags Test tags are a great way of organizing tests outside of a file structure. To learn more see the official documentation [here](https://playwright.dev/docs/test-annotations#tag-tests). Current list of test tags: |Test Tag|Description| |:-:|-| |`@mobile` | Test case or test suite is compatible with Playwright's iPad support and Open MCT's read-only mobile view (i.e. no create button).| |`@a11y` | Test case or test suite to execute playwright-axe accessibility checks and generate a11y reports.| |`@addInit` | Initializes the browser with an injected and artificial state. Useful for loading non-default plugins. Likely will not work outside of `npm start`.| |`@localStorage` | Captures or generates session storage to manipulate browser state. Useful for excluding in tests which require a persistent backend (i.e. CouchDB). See [note](#utilizing-localstorage)| |`@snapshot` | Uses Playwright's snapshot functionality to record a copy of the DOM for direct comparison. Must be run inside of the playwright container.| |`@2p` | Indicates that multiple users are involved, or multiple tabs/pages are used. Useful for testing multi-user interactivity.| |`@generatedata` | Indicates that a test is used to generate testdata or test the generated test data. Usually to be associated with localstorage, but this may grow over time.| |`@clock` | A test which modifies the clock. These have expanded out of the visual tests and into the functional tests. |`@framework` | A test for open mct e2e capabilities. This is primarily to ensure we don't break projects which depend on sourcing this project's fixtures like appActions.js. ### Continuous Integration The cheapest time to catch a bug is pre-merge. Unfortunately, this is the most expensive time to run all of the tests since each merge event can consist of hundreds of commits. For this reason, we're selective in _what we run_ as much as _when we run it_. We leverage Github Actions / Workflows to execute tests as it gives us the ability to run against multiple operating systems with greater control over git event triggers (i.e. Run on a PR Comment event). Our CI environment consists of 3 main modes of operation: #### 1. Per-Commit Testing GitHub Actions - e2e tests against ubuntu and chrome - Performance tests against ubuntu and chrome - e2e tests are linted - Visual and a11y tests are run in a single resolution on the default `espresso` theme #### 2. Per-Merge Testing Github Actions - Full suite against all browsers/projects. Triggered with Github Label Event 'pr:e2e' - CouchDB Tests. Triggered on PR Create and again with Github Label Event 'pr:e2e:couchdb' #### 3. Scheduled / Batch Testing Nightly Testing in Github Action - Full e2e suite against ubuntu and chrome, firefox, and an MMOC resolution profile - Performance tests against ubuntu and chrome - CouchDB suite - Visual and a11y Tests are run in the full profile #### Parallelism and Fast Feedback In order to provide fast feedback in the Per-Commit context, we try to keep total test feedback at 5 minutes or less. That is to say, A developer should have a pass/fail result in under 5 minutes. Playwright has native support for semi-intelligent sharding. Read about it [here](https://playwright.dev/docs/test-parallel#shard-tests-between-multiple-machines). We will be adjusting the parallelization of the Per-Commit tests to keep below the 5 minute total runtime threshold. In addition to the Parallelization of Test Runners (Sharding), we're also running four concurrent threads on every shard. So for every commit, Playwright is effectively running 4 x 2 concurrent browsercontexts to keep the overall runtime to a miminum. At the same time, we don't want to waste CI resources on parallel runs, so we've configured each shard to fail after 5 test failures. Test failure logs are recorded and stored to allow fast triage. ### Cross-browser and Cross-operating system #### **What's supported:** We are leveraging the `browserslist` project to declare our supported list of browsers. We support macOS, Windows, and ubuntu 20+. #### **Where it's tested:** We lint on `browserslist` to ensure that we're not implementing deprecated browser APIs and are aware of browser API improvements over time. We also have the need to execute our e2e tests across this published list of browsers. Our browsers and browser version matrix is found inside of our `./playwright-*.config.js`, but mostly follows in order of bleeding edge to stable: - `playwright-chromium channel:beta` - A beta version of Chromium from official chromium channels. As close to the bleeding edge as we can get. - `playwright-chromium` - A stable version of Chromium from the official chromium channels. This is always at least 1 version ahead of desktop chrome. - `playwright-chrome` - The stable channel of Chrome from the official chrome channels. This is always 2 versions behind chromium. - `playwright-firefox` - Firefox Latest Stable. Modified slightly by the playwright team to support a CDP Shim. In terms of operating system testing, we're only limited by what the CI providers are able to support. The bulk of our testing is performed on the official playwright container which is based on ubuntu. Github Actions allows us to use `windows-latest` and `mac-latest` and is run as needed. #### **Mobile** We have a Mission-need to support iPad and mobile devices. To run our test suites with mobile devices, please see our `playwright-mobile.config.js` projects. In general, our test suite is not designed to run against mobile devices as the mobile experience is a focused version of the application. Core functionality is missing (chiefly the 'Create' button). To bypass the object creation, we leverage the `storageState` properties for starting the mobile tests with localstorage. For now, the mobile tests will exist in the /tests/mobile/ suites and be executed with the ```sh npm run test:e2e:mobile ``` command. #### **Skipping or executing tests based on browser, os, and/os browser version:** Conditionally skipping tests based on browser (**RECOMMENDED**): ```js test('Can adjust image brightness/contrast by dragging the sliders', async ({ page, browserName }) => { // eslint-disable-next-line playwright/no-skipped-test test.skip(browserName === 'firefox', 'This test needs to be updated to work with firefox'); // ... ``` Conditionally skipping tests based on OS: ```js test('Can adjust image brightness/contrast by dragging the sliders', async ({ page }) => { // eslint-disable-next-line playwright/no-skipped-test test.skip(process.platform === 'darwin', 'This test needs to be updated to work with MacOS'); // ... ``` Skipping based on browser version (Rarely used): ## Test Design, Best Practices, and Tips & Tricks ### Test Design #### Test as the User In general, strive to test only through the UI as a user would. As stated in the [Playwright Best Practices](https://playwright.dev/docs/best-practices#test-user-visible-behavior): > "Automated tests should verify that the application code works for the end users, and avoid relying on implementation details such as things which users will not typically use, see, or even know about such as the name of a function, whether something is an array, or the CSS class of some element. The end user will see or interact with what is rendered on the page, so your test should typically only see/interact with the same rendered output." By adhering to this principle, we can create tests that are both robust and reflective of actual user experiences. #### How to make tests robust to function in other contexts (VISTA, COUCHDB, YAMCS, VIPER, etc.) 1. Leverage the use of `appActions.js` methods such as `createDomainObjectWithDefaults()`. This ensures that your tests will create unique instances of objects for your test to interact with. 1. Do not assert on the order or structure of objects available unless you created them yourself. These tests may be used against a persistent datastore like couchdb with many objects in the tree. 1. Do not search for your created objects. Open MCT does not performance uniqueness checks so it's possible that your tests will break when run twice. 1. Avoid creating locator aliases. This likely means that you're compensating for a bad locator. Improve the application instead. 1. Leverage `await page.goto('./', { waitUntil: 'domcontentloaded' });` instead of `{ waitUntil: 'networkidle' }`. Tests run against deployments with websockets often have issues with the networkidle detection. #### How to make tests faster and more resilient to application changes 1. Avoid app interaction when possible. The best way of doing this is to navigate directly by URL: ```js // You can capture the CreatedObjectInfo returned from this appAction: const clock = await createDomainObjectWithDefaults(page, { type: 'Clock' }); // ...and use its `url` property to navigate directly to it later in the test: await page.goto(clock.url); ``` 1. Leverage `await page.goto('./', { waitUntil: 'domcontentloaded' });` - Initial navigation should _almost_ always use the `{ waitUntil: 'domcontentloaded' }` option. 1. Avoid repeated setup to test a single assertion. Write longer tests with multiple soft assertions. This ensures that your changes will be picked up with large refactors. 1. Use [user-facing locators](https://playwright.dev/docs/best-practices#use-locators) (Now a eslint rule!) ```js page.getByRole('button', { name: 'Create' } ) ``` Instead of ```js page.locator('.c-create-button') ``` Note: `page.locator()` can be used in performance tests as xk6-browser does not yet support the new `page.getBy` pattern and css lookups can be [1.5x faster](https://serpapi.com/blog/css-selectors-faster-than-getbyrole-playwright/) ##### Utilizing LocalStorage 1. In order to save test runtime in the case of tests that require a decent amount of initial setup (such as in the case of testing complex displays), you may use [Playwright's `storageState` feature](https://playwright.dev/docs/api/class-browsercontext#browser-context-storage-state) to generate and load localStorage states. 1. To generate a localStorage state to be used in a test: - Add an e2e test to our generateLocalStorageData suite which sets the initial state (creating/configuring objects, etc.), saving it in the `test-data` folder: ```js // Save localStorage for future test execution await context.storageState({ path: path.join(__dirname, '../../../e2e/test-data/display_layout_with_child_layouts.json') }); ``` - Load the state from file at the beginning of the desired test suite (within the `test.describe()`). (NOTE: the storage state will be used for each test in the suite, so you may need to create a new suite): ```js const LOCALSTORAGE_PATH = path.resolve( __dirname, '../../../../test-data/display_layout_with_child_layouts.json' ); test.use({ storageState: path.resolve(__dirname, LOCALSTORAGE_PATH) }); ``` ### How to write a great test - Avoid using css locators to find elements to the page. Use modern web accessible locators like `getByRole` - Use our [App Actions](./appActions.js) for performing common actions whenever applicable. - Use `waitForPlotsToRender()` before asserting against anything that is dependent upon plot series data being loaded and drawn. - If you create an object outside of using the `createDomainObjectWithDefaults` App Action, make sure to fill in the 'Notes' section of your object with `page.testNotes`: ```js // Fill the "Notes" section with information about the // currently running test and its project. const { testNotes } = page; const notesInput = page.locator('form[name="mctForm"] #notes-textarea'); await notesInput.fill(testNotes); ``` #### How to Write a Great Visual Test 1. **Look for the Unknown Unknowns**: Avoid asserting on specific differences in the visual diff. Visual tests are most effective for identifying unknown unknowns. 2. **Get the App into Interesting States**: Prioritize getting Open MCT into unusual layouts or behaviors before capturing a visual snapshot. For instance, you could open a dropdown menu. 3. **Expect the Unexpected**: Use functional expect statements only to verify assumptions about the state between steps. A great visual test doesn't fail during the test itself, but rather when changes are reviewed in Percy.io. 4. **Control Variability**: Account for variations inherent in working with time-based telemetry and clocks. - Utilize `percyCSS` to ignore time-based elements. For more details, consult our [percyCSS file](./.percy.ci.yml). - Use Open MCT's fixed-time mode unless explicitly testing realtime clock - Employ the `createExampleTelemetryObject` appAction to source telemetry and specify a `name` to avoid autogenerated names. - Avoid creating objects with a time component like timers and clocks. - Utilize the playwright clock() API. See @clock Annotations for examples. 5. **Hide the Tree and Inspector**: Generally, your test will not require comparisons involving the tree and inspector. These aspects are covered in component-specific tests (explained below). To exclude them from the comparison by default, navigate to the root of the main view with the tree and inspector hidden: - `await page.goto('./#/browse/mine?hideTree=true&hideInspector=true')` 6. **Component-Specific Tests**: If you wish to focus on a particular component, use the `/visual-a11y/component/` folder and limit the scope of the comparison to that component. For instance: ```js await percySnapshot(page, `Tree Pane w/ single level expanded (theme: ${theme})`, { scope: treePane }); ``` - Note: The `scope` variable can be any valid CSS selector. 7. **Write many `percySnapshot` commands in a single test**: In line with our approach to longer functional tests, we recommend that many test percySnapshots are taken in a single test. For instance: ```js // await percySnapshot(page, `Before object expanded (theme: ${theme})`); // await percySnapshot(page, `object expanded (theme: ${theme})`); //Select from object await percySnapshot(page, `object selected (theme: ${theme})`) ``` 8. **Use `networkidle` to wait for network requests to complete**: This is necessary to ensure that all network requests have completed before taking a snapshot. This ensures that icons are loaded and other assets are available. https://github.com/nasa/openmct/issues/7549 #### How to write a great network test - Where possible, it is best to mock out third-party network activity to ensure we are testing application behavior of Open MCT. - It is best to be as specific as possible about the expected network request/response structures in creating your mocks. - Make sure to only mock requests which are relevant to the specific behavior being tested. - Where possible, network requests and responses should be treated in an order-agnostic manner, as the order in which certain requests/responses happen is dynamic and subject to change. Some examples of mocking network responses in regards to CouchDB can be found in our [couchdb.e2e.spec.js](./tests/functional/couchdb.e2e.spec.js) test file. ### Best Practices For now, our best practices exist as self-tested, living documentation in our [exampleTemplate.e2e.spec.js](./tests/framework/exampleTemplate.e2e.spec.js) file. For best practices with regards to mocking network responses, see our [couchdb.e2e.spec.js](./tests/functional/couchdb.e2e.spec.js) file. ### Tips & Tricks The following contains a list of tips and tricks which don't exactly fit into a FAQ or Best Practices doc. - (Advanced) Overriding the Browser's Clock It is possible to override the browser's clock in order to control time-based elements. Since this can cause unwanted behavior -- i.e. Tree not rendering -- only use this sparingly. Use the `page.clock()` API as such: ```js import { test, expect } from '../../pluginFixtures.js'; test.describe('foo test suite @clock', () => { test.beforeEach(async ({ page }) => { //Set clock time await page.clock.install({ time: MISSION_TIME }); await page.clock.resume(); //Navigate to page with new clock await page.goto('./', { waitUntil: 'domcontentloaded' }); }); test('bar here', async ({ page }) => { /// ... }); ``` - Working with multiple pages There are instances where multiple browser pages will needed to verify multi-page or multi-tab application behavior. Make sure to use the `@2p` annotation as well as name each page appropriately: i.e. `page1` and `page2` or `tab1` and `tab2` depending on the intended use case. Generally pages should be used unless testing `sharedWorker` code, specifically. - Working with file downloads and JSON data Open MCT has the capability of exporting certain objects in the form of a JSON file handled by the chrome browser. The best example of this type of test can be found in the exportAsJson test. ```js const [download] = await Promise.all([ page.waitForEvent('download'), // Waits for the download event page.getByLabel('Export as JSON').click() // Triggers the download ]); // Wait for the download process to complete const path = await download.path(); // Read the contents of the downloaded file using readFile from fs/promises const fileContents = await fs.readFile(path, 'utf8'); const jsonData = JSON.parse(fileContents); // Use the function to retrieve the key const key = getFirstKeyFromOpenMctJson(jsonData); // Verify the contents of the JSON file expect(jsonData.openmct[key]).toHaveProperty('name', 'e2e folder'); ``` ### Reporting Test Reporting is done through official Playwright reporters and the CI Systems which execute them. We leverage the following official Playwright reporters: - HTML - junit - github annotations - Tracefile - Screenshots When running the tests locally with the `npm run test:e2e:local` command, the html report will open automatically on failure. Inside this HTML report will be a complete summary of the finished tests. If the tests failed, you'll see embedded links to screenshot failure, execution logs, and the Tracefile. When looking at the reports run in CI, you'll leverage this same HTML Report which is hosted in Github Actions as a build artifact. ### e2e Code Coverage Our e2e code coverage is captured and combined with our unit test coverage. For more information, please see our [code coverage documentation](../TESTING.md) #### Generating e2e code coverage Please read more about our code coverage [here](../TESTING.md#code-coverage) ## Other ### About e2e testing e2e testing is an industry-standard approach to automating the testing of web-based UIs such as Open MCT. Broadly speaking, e2e tests differentiate themselves from unit tests by preferring replication of real user interactions over execution of raw JavaScript functions. Historically, the abstraction necessary to replicate real user behavior meant that: - e2e tests were "expensive" due to how much code each test executed. The closer a test replicates the user, the more code is needed run during test execution. Unit tests could run smaller units of code more efficiently. - e2e tests were flaky due to network conditions or the underlying protocols associated with testing a browser. - e2e frameworks relied on a browser communication standard which lacked the observability and controls necessary needed to reach the code paths possible with unit and integration tests. - e2e frameworks provided insufficient debug information on test failure However, as the web ecosystem has matured to the point where mission-critical UIs can be written for the web (Open MCT), the e2e testing tools have matured as well. There are now fewer "trade-offs" when choosing to write an e2e test over any other type of test. Modern e2e frameworks: - Bypass the surface layer of the web-application-under-test and use a raw debugging protocol to observe and control application and browser state. - These new browser-internal protocols enable near-instant, bi-directional communication between test code and the browser, speeding up test execution and making the tests as reliable as the application itself. - Provide test debug tooling which enables developers to pinpoint failure Furthermore, the abstraction necessary to run e2e tests as a user enables them to be extended to run within a variety of contexts. This matches the extensible design of Open MCT. A single e2e test in Open MCT is extended to run: - Against a matrix of browser versions. - Against a matrix of OS platforms. - Against a local development version of Open MCT. - A version of Open MCT loaded as a dependency (VIPER, VISTA, etc) - Against a variety of data sources or telemetry endpoints. ### Why Playwright? [Playwright](https://playwright.dev/) was chosen as our e2e framework because it solves a few VIPER Mission needs: 1. First-class support for Automated Performance Testing 2. Official Chrome, Chrome Canary, and iPad Capabilities 3. Support for Browserless.io to run tests in a "hermetically sealed" environment 4. Ability to generate code coverage reports ### FAQ - How does this help NASA missions? - When should I write an e2e test instead of a unit test? - When should I write a functional vs visual test? - How is Open MCT extending default Playwright functionality? - What about Component Testing? ### Writing Tests Playwright provides 3 supported methods of debugging and authoring tests: - A 'watch mode' for running tests locally and debugging on the fly - A 'debug mode' for debugging tests and writing assertions against tests - A 'VSCode plugin' for debugging tests within the VSCode IDE. Generally, we encourage folks to use the watch mode and provide a script `npm run test:e2e:watch` which launches the launch mode ui and enables hot reloading on the dev server. ### e2e Troubleshooting Please follow the general guide troubleshooting in [the general troubleshooting doc](../TESTING.md#troubleshooting-ci) - Tests won't start because 'Error: # is already used...' This error will appear when running the tests locally. Sometimes, the webserver is left in an orphaned state and needs to be cleaned up. To clear up the orphaned webserver, execute the following from your Terminal: ```lsof -n -i4TCP:8080 | awk '{print$2}' | tail -1 | xargs kill -9``` ### Upgrading Playwright In order to upgrade from one version of Playwright to another, the version should be updated in several places in both `openmct` and `openmct-yamcs` repos. An easy way to identify these locations is to search for the current version in all files and find/replace. For reference, all of the locations where the version should be updated are listed below: #### **In `openmct`:** - `package.json` - Both packages `@playwright/test` and `playwright-core` should be updated to the same target version. - `.github/workflows/ci.yml` - `.github/workflows/e2e-couchdb.yml` - `.github/workflows/e2e-pr.yml` #### **In `openmct-yamcs`:** - `package.json` - `@playwright/test` should be updated to the target version. - `.github/workflows/yamcs-quickstart-e2e.yml` --- ## File: example/README.md This directory is for example bundles, which are intended to illustrate how to author new software components using Open MCT. ## 2. Official Technical Reference & Guides (nasa/nasa.github.io) ## File: README.md # NASA's Public GitHub Organization --- ## Purpose The [github.com/nasa](https://github.com/nasa/) organizational account is intended to publicly host NASA code that has been SRA-approved (Software Release Authority) for open source. --- ## Instructions **If you are looking for information & instructions on [NASA's Public GitHub Organization](https://github.com/nasa), Please check out [INSTRUCTIONS.md](/docs/INSTRUCTIONS.md)** Please reach out to us at hq-open-innovation@mail.nasa.gov if you have any questions not covered in the instructions above. --- #### Where old site went to? Please note that active development has shifted away from the website and into the markdown file https://github.com/nasa/instructions/blob/master/docs/INSTRUCTIONS.md --- --- ## File: docs/about_section_instructions.md # BEST PRACTICES FOR REPOSITORY NAMES AND ABOUT SECTION INFORMATION ON GITHUB.COM/NASA ### Context: Where These Instructions Apply There are instructions for the public-facing GitHub! Specifically the NASA org at https://github.com/nasa These instructions are not for any internal GitHub or GitLab instance. More general instructions for Open Sourcing Code are on [https://code.nasa.gov/#/guide](https://code.nasa.gov/#/guide) More general Instructions for Github.com/nasa are on [https://github.com/nasa/instructions/blob/master/docs/INSTRUCTIONS.md](https://github.com/nasa/instructions/blob/master/docs/INSTRUCTIONS.md) **/ ### Certain Information Can Only Populated By Admins on GitHub.com/nasa To comply with requests from security and legal that all repositories accessible to the public on github.com/nasa have an SRA authorization number, repositories can only be created by administrators of github.com/nasa. This means only administrators can "own" repositories using GitHub definitions. In addition to limiting who can "create" repositories, ownership also limits who can populate: - repository name - topic tags - website link - description #### This means all those fields need to be submitted in your NAMs request, or in an email to an admin, when requesting a repository!!! *More information on them is below.* ## Repository Names It is difficult to make absolute rules when discussing code repositories. What follows are more guidelines. #### Changing a repository name is the easiest right when you first releasing it as open source! Changing a repository's name during use can cause configuration & link issues. Changing a name of a code repository as it is being released for open source is probably the easiest time to change it after creation. Open-sourcing is also a point in time when the audience for the code is changing. The name that made sense and was functional for your small internal team may not be a name with the same understanding and value to the general public. #### Goals of a code repository name Ideally, a code repository name would convey meaning to the potential future end user about its purpose, be easily distinguishable from other related code repositories, and be memorable. In the worst case, a name can cause confusion about what it does, be hard distinguish from related code, and be impossible to remember the name. A long name is actually not that much of a negative when it comes to code repositories as developers typically use tab-complete to finish the name anyways. ##### Related Repositories For a code project that involves multiple different repositories, it is good to name them such that the shared concept is at the front. This helps for both search and discovery. This might look like: ``` projectname-coderepositoryname ``` or sometimes there may be a need for a third breakdown to convey the difference between an application build for one operating system or computer language vs. another. For example: ``` venusweather-api-python ``` vs. ``` venusweather-api-go ``` ##### Syntax There isn't a very strict standard in terms of syntax of repository names but in general making them easier to read is better. This leads to the following practices: - **dashes > underscores ** - **dashes-dashes > CamelCase ** - **lowercase > UPPERCASE** #### Acronyms are bad names, usually There is a tendancy in government to name everything that needs a name with an acronym. Acronyms save time for those that know their meaning, but they convey somewhere between very little and zero information to the people who don't know their meaning. As such, if there is a choice between using three words separated by dashes and an acronym, the three words separated by dashes is nearly always the better name. You can always explain in the README that internally, the project is referred to as acronym ___. ## Topic tags improve code discoverability Topic tags help the public discover your code in a variety of ways. - The public might search for the topic tag across all of GitHub. - The public might be on a related code repository on github.com/nasa that uses the same tag. They an click on the tag and see any code repositories under the NASA org with the same tag. - Tags also helps NASA individuals understand what types of NASA code has been open-sourced and is available for reuse. Good examples of topic tags are domains, fields of study, frameworks used, public & internal project names, NASA org names, and especially single words that describe possible software use cases in general terms. For examples of good tags, you can look at the tags on code.nasa.gov https://code.nasa.gov/ . Below each code project description is a list of both human populated tags and machine-learning generated tags. When you submit a new project to be cataloged on code.nasa.gov, [which is mandatory for all open-source release](https://code.nasa.gov/#/guide), it triggers a pipeline that generates the machine-learning generated tags. ### Required Tags For Any Machine Learning Related Code A.I. Related Datasets Need New OMB Mandated Tags OMB has issued guidance ["Guidance for Improving Discoverability of Agency Datasets for AI R&D"](https://code.gov/assets/data/ai_inventory-guidance.pdf). If your dataset in any way relates to A.I., please read the linked instructions and add the appropriate tags! ``` Agencies shall include the keyword of “usg-artificial-intelligence” for all datasets determined to support AI R&D. Datasets that specifically serve as training data for ML applications should additionally include a keyword of “usg-ai-training-data,” “usg-ai-testing-data,” ``` You can find all the NASA repositories that are related to A.I. and use this tag [here](https://github.com/orgs/nasa/repositories?language=&q=usg-artificial-intelligence&sort=&type=). This topic tag is relatively new, so hopefully this list will expand in size over time. ## Website Link If you want to use GitHub pages to display documentation, please request that the link to the github pages be set up and supplied in the "website" field. Alternatively, if there is no documentation website but the code was created for a NASA mission or project that does have a website, please consider adding it in that field. ## Description Many NASA code repositories on github.com/nasa lack a description or have a description that still leaves a members of the public unsure what exactly the code does. Please write the description with someone not familiar with the code, your project, or NASA in mind. This is how much text you have available, which is approximately 350 characters with spaces. ``` Information about the NASA Github organization is on nasa.github.io/ Information about the NASA Github organization is on nasa.github.io/ Information about the NASA Github organization is on nasa.github.io/ Information about the NASA Github organization is on nasa.github.io/ Information about the NASA Github organization is on nasa.github.io/ In ``` --- ## File: docs/INSTRUCTIONS.md # INSTRUCTIONS FILE FOR NASA's PUBLIC GITHUB ORGANIZATION ## Table of Contents - [Purpose](#purpose) - [Instructions for Who Can Be a Member](#instructions-for-who-can-be-a-member) - [Instructions for Once You Are a Member](#instructions-for-once-you-are-a-member) - [Use of Large File Storage](#use-of-large-file-storage) - [Use of GitHub Actions](#use-of-github-actions) - [GitHubPages and Websites](#githubpages-and-websites) - [Enabling Discussions and Other Beta Features](#enabling-discussions-and-other-beta-features) - [Most Common Requests](#most-common-requests) - [Related Sites](#related-sites) - [Questions](#questions) ## Purpose This GitHub organization is intended to publicly host NASA code that has been SRA-approved for open source. You can review the SRA (software release authority) process for getting open-source approval on [https://code.nasa.gov/#/guide](https://code.nasa.gov/#/guide). ## Instructions for Who Can Be a Member Only NASA staff can be a member of the NASA org on github.com and release code in [github.com/nasa](https://github.com/nasa) org. If you are a NASA civil servant or contractor and would like to be added to the Public NASA GitHub Org, you will need to submit the AGCY NASA Github Collaborator Form [NAMS request ](https://nams.nasa.gov) https://nams.nasa.gov. This requires you to be behind the NASA firewall. If you were a member several years ago and recently found you are no longer a member, please see this [note](https://github.com/nasa/nasa.github.io/issues/2). Only members have rights to directly edit code in a [github.com/nasa](https://github.com/nasa) repository. Everyone else will need to submit a pull request. We highly encourage pull requests from the public!!! However, please note that not all repositories are actively maintained. If you look at NASA code repositories on the [https://github.com/nasa](https://github.com/nasa) front page, there is a little sparkline (small inline chart) showing activity over time. Projects with lots of activity are more likely to quickly accept a pull request. Members of the public can not attain and do not need collaborator status. *You don't need any special status to submit pull requests or add issues.* For information on how to become a member of the NASA Org (you need to be a NASA civil servant or contractor with access to systems inside the NASA firewall), please see the instructions on [code.nasa.gov/#/guide](https://code.nasa.gov/#/guide). ### Instructions for Once You Are a Member *Please read these instructions in full as using github.com/nasa has some differences with how you might be familar using github.com on personal projects.* To ensure mandatory policies are followed, NASA org members are not given the full admin rights you might be used to on your personal github.com accounts. However, by using teams, team owners, collaborators, and outside collaborators roles we can make the experience pretty quick and easy. #### Org Owners We currently have six active owners in the org: Jennifer Wei: github username = jwei-openscapes Darith Yim : github username = dsyim Daniel Rendon : github username = dcrendon Alexander R Kirtley : github username = AlexKirtleyNASA Andre Gueret: github username = AndreGueretNASA A general purpose contact email is [hq-open-innovation@mail.nasa.gov](mailto:hq-open-innovation@mail.nasa.gov), please mention the specific nature of your question as this email is also used on data.nasa.gov & api.nasa.gov. **Only org owners have the ability to create new repositories, bring new users in as members of the NASA github org, and make changes to settings of repositories**. If there is something a member of the nasa org can not do that this arragement prevents, please reach out. #### Repos We do not give out admin rights on any NASA Github repos for reasons stated above. If you want a new repo created or want to edit settings on an existing repo that require admin access, please contact one of our org owners. Without admin rights on a repository, you'll still be able to make any code changes yourself as long as you're a collaborator on that repository or a member of a team whose participants are collaborators by default. ##### Required Information When Requesting New Repository If you need for a new repository (and it has gotten SRA approval) please either submit a modification to your NAMs request with the following information: - **Name of the repository** - About Tab Information: These will appear on the repo's home page in the "about" section. They are extractable as metadata and assist in discoverability. - **Topic tags** that describe the repository. - **A 350 character or less description** of the repository. - **Website Link** : If you intend to start up a github pages site for documentation or have a link that goes elsewhere with that information, supply it for the about section as well. - **SRA number** - **Team name** for managing the repository - **Usernames of people for that team** if it doesn't already exist. Usernames must be from the list of existing NASA org members. [PLEASE READ THESE INSTRUCTIONS FOR A LONGER DESCRIPTION OF BEST PRACTICES FOR REPOSITORY NAMES AND ABOUT SECTION INFORMATION](https://github.com/nasa/nasa.github.io/blob/master/docs/about_section_instructions.md) If you are joining for the first time and need a repository created, you can also include the same information in your NAMs request for access inside the more information box. You can learn more about the software release authority process on the [guide page](https://code.nasa.gov/#/guide) of code.nasa.gov. #### Teams Teams are effective ways of managing repository access privileges for an entire group of users. Only org owners can create and remove teams, but we now allow users to be team maintainers. Team maintainers have permissions to add and remove users to their teams from the population of people who are already NASA org members. Note: Team maintainers are technically able to add a user who was once in the NASA org back onto their team (thus reinstating the user in the org). However, we do not allow this and will remove maintainers who are found in violation. If you need a past member to be reinstated in the org, contact an org owner and request to be added as an outside collaborator. #### Collaborators Github also has a role called "collaborator". This is someone who doesn't have repository ownership or admin rights but can push edits to code directly without doing a pull request. If you would like to add a NASA staff member who is a member of the NASA org as a collaborator on a single repository, we still suggest you contact the NASA org owners mentioned above about setting up a TEAM for that single repository instead of adding people one by one. Teams result in less requestes (and therefore waiting) on NASA org owners for actions. We are sorry you don't have normal full ownership rights. It is because everyone with write access needs to be approved NASA user. If you had normal repository owner right, you could add any github user. #### Outside Collaborators Only on a rare case-by-case basis we will allow non-NASA users to be added to the org as Outside Collaborators. You can collaborate without this status via forks, branches, and pull-requests. If you fork a repository, you can make changes and then submit back a pull-request. A repository maintainer can then approve those changes. You do not need formal outside-collaborator status for this! Please reach out to us if you have any questions. ### Use of Large File Storage NASA is on a free-tier Github organization account that has major limitations for the Git LFS service (maximum of 10GB for both monthly storage and monthly bandwidth which resets the 1st of each month), so we do not recommend using Git LFS tracking large files in your repositories. Please ensure the removal of all large files from your repository and commit history. You will need to plan to host your large files on your team’s end through another available service. ### Use of GitHub Actions NASA is on a free-tier Github organization account that has limitations for the Github Actions (2,000 CI/CD minutes/month and FREE for public repositories). You can use execution minutes with GitHub Actions to automate your software development workflows (write tasks and combine them to build, test, and deploy any code project on GitHub). #### Billing Plan & Impact on Features A very small number of GitHub features depend on what billing plan your organization uses. This varies over time, so please refer to GitHub documentation for this information. Github.com/nasa is on a "legacy" billing plan until October 1st, 2022 at which time it switches to a “free” plan. Free plan has costs for GitHubActions and Package storage, so there may be a future point in time when those features would be curtailed for the heaviest users. Any change would be announced. GitHub.com/nasa is not on an Enterprise plan as it would trigger internal policies that would require payment at the individual level resulting in less open source code released. ### GitHubPages and Websites You must contact an administrator to help you turn on GitHub pages. Please let them know the branch being used and whether the starting `index.html` file is under the top level of the repository or the `docs` folder. *PLEASE NOTE: GitHub pages can be used for documentation of NASA open source code. You can not use GitHub pages to stand up a NASA website as it will not be under the nasa.gov domain.* Please contact the github.com/nasa administrators listed elsewhere on this page if you have any questions about the interpretation of this policy. ##### Why is this now just a markdown file instead of a website? These instructions were previously made into a website. The fact that it looked like a website, not documentation, and lacked a nasa.gov domain caused it to attract negative attention. Hence, it was reverted back into a markdown file document format. Apologies to those who submitted pull requests to improve the old site. ### Enabling Discussions and Other Beta Features There are some repository configurations, especially for beta features of GitHub, that are not enabled by default. These often need to be enabled in the settings menu, which is not always available for users who only have write access. If there is a feature you see elsewhere in GitHub.com that you don't see enabled on your repository, please reach out to the administrator email. We will see if that is a feature we can enable for you. One example, would be "discussions", which is a place to have discussions that don't fall squarely within "issues" or "pull requests". Reach out to the contact email to get "discussions" enabled on a NASA repository that you can write access to. Include your name, email, GitHub username, and repository name. ### Most Common Requests
Need: *If you need to be added to the NASA org on github.com/* - Create a personal github account and then request to be added as a Github Collaborator via a new or modified [NAMs request](https://idmax.nasa.gov/nams/asset/227756). Please include your github id in the request. Please note that you will not be able to get to that link if you are not inside the NASA firewall.
--->THIS IS THE MOST COMMON REQUEST<--- Need: *If you need to add a new open-sourced repository to github.com/nasa that has already been approved via the software release authority process.* - Make or modify a request to start a new repository in github.com/nasa via the [NAMs request](https://idmax.nasa.gov/nams/asset/227756). Please give it a useful name and good documentation! We'll create a blank repository for you to put the code into. Please note that you will not be able to get to that link if you are not inside the NASA firewall. In your NAMs request, please include all the required metadata describing the respository listed in the "REPO" section above. If this is the first repository you're open sourcing, you might also want to read through the longer description of how to supply the metadata [here](https://github.com/nasa/instructions/blob/master/docs/about_section_instructions.md).
Need: *If you need to create a new team for managing who has access to make changes on a particular repository or group of repositories.* - Modify your [NAMs request](https://idmax.nasa.gov/nams/asset/227756) to request the team creation or email the two owners above. Give team name and which repositories team members should be given access to. Please note that you will not be able to get to that link if you are not inside the NASA firewall.
Need: *If you need to add a new member to an already established team.* - Send an email to the Team owner, modify your NAMs request, or email the Github Owners in that order of preference.
Need: *I accidentally committed something that needs to be deleted from all git history. How do I do that?* - Check out the instructions here: https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/removing-sensitive-data-from-a-repository#using-the-bfg
Need: *Someone submitted a spam pull request to a NASA repository. That user needs blocked from all NASA repositories. * - Send an email to an NASA org admin using the standard contact email listed on this page. Supply a link to the issue or pull request with the spam and the username of who is to be blocked.
## Related Sites ### Organizationally Related Sites In addition to github.com/nasa, NASA OCIO's Transformation and Data Division Open-Innovation staff also run: - [code.nasa.gov](https://code.nasa.gov) [Catalog containing metadata describing open source NASA code that is on github.com/nasa as well as other locations.] - [data.nasa.gov](https://data.nasa.gov) [Catalog that holds metadata that describes open datasets from many different NASA data archives.] - [api.nasa.gov](https://api.nasa.gov) [Listing of beginner friendly NASA APIs.] ### Open Source Related Pages on NASA.GOV The page https://www.nasa.gov/open/open-source-development.html has some high level history and information regarding open source at NASA. ### Information on Internal Code Platforms [Internal information](https://wiki.jsc.nasa.gov/fod/index.php/Knowledge_Any_Developer_New_to_NASA_Will_Need_to_Know) on internal code platforms ### Why is NASA's Open Source Code in multiple locations? The reason for NASA open source code existing in different places boils down to different requirements, both requirements of those writing the code and requirements from above. Other NASA orgs on github.com or gitlab.com or bitbucket Although NASA staff are highly encouraged to release open source software on github.com/nasa as that is where most people look so it ensures higher potential reuse, they are not absolutely required to use that org account. A common reason code may exist under other orgs is that the software is jointly maintained by non-NASA persons or the code was developed by non-NASA persons through a NASA grant. To encourage disoverability, please use the #NASA topic tag on any GitHub or GitLab repositories not under github.com/nasa. Doing so helps create views of NASA open source code beyond what is under github.com/nasa. For example, [this webpage creates visualizations of trends and connections between NASA repositories](https://justingosses.github.io/nasa-repos-visual-explorer/explore/) across many org accounts and not just github.com/nasa. ### Discovering useful NASA Open-Source Code via metadata catalogs There are several hundred NASA open-source code repositories and as previously noted, not everything is on github.com/nasa. To improve your chance of finding something useful to you, we suggest you use code.nasa.gov and software.nasa.gov. These sites host metadata descriptions for nearly all of the officially open-sourced code. #### Code.nasa.gov Code.nasa.gov feeds into https://code.gov, which holds open source code from across all federal government agencies. It is mandated by Congress and only holds standardized metadata about the code. Code.nasa.gov only shows open-source software. For government-source and patented software, see software.nasa.gov Please make sure any repos added to github.com/nasa are also tracked in code.nasa.gov! Code.nasa.gov uses both human and A.I. generated tags to help users find NASA open source that is useful for their goals. Screenshot of NASA's project OPEN MCT #### Software.nasa.gov Software.nasa.gov is part of the larger https://technology.nasa.gov/ that also covers patents and spinoffs. The NASA's Technology Transfer Program ensures that innovations developed for exploration and discovery are broadly available to the public, maximizing the benefit to the Nation. Whether you're looking to start a new company, enhance an existing product, or create a new product line, you can gain a competitive edge in the marketplace by putting NASA technology to work for you. The information on software.nasa.gov is presented in a way that leverages NASA's technology taxonomy, so if you want to see an aggregate view of NASA software categories, it is a good place to visit. It also includes some older software products that are only available as downloadable zip files as well as software that must be requested individually as they are only open to US persons or are tied to a patent. Screenshot of NASA's software categories ## Questions If you have questions, you may leave an issue on this repository or contact us via [hq-open-innovation@mail.nasa.gov](mailto:hq-open-innovation@mail.nasa.gov). We will use those questions & answers to eventually make a FAQ section on this page.