Summary and Setup

Neutrons Open Visualization and Analysis Framework Tutorial

This tutorial guides you through the process of building applications for neutron data analysis using the NOVA framework. NOVA provides key components to create applications that range from simple scripts, all the way to full user interfaces with advanced visualization capabilities. It leverages a Model-View-ViewModel (MVVM) architectural pattern and three key NOVA libraries: nova-galaxy, nova-trame, and nova-mvvm, to simplify interaction with the Neutron Data Interpretation Platform (NDIP).

Key Learning Objectives:

  • Introduction to NDIP and NOVA: Understand the purpose of NDIP as a workflow management system for neutron data and NOVA as a framework for simplifying NDIP application development.
  • Project Setup: Learn how to use the copier library to clone a template application, providing a basic project structure and pre-configured files.
  • NDIP Interaction with nova-galaxy: Master the use of nova-galaxy to connect to NDIP, define and run tools, and manage input parameters.
  • MVVM Implementation with nova-mvvm and Pydantic: Grasp the principles of the MVVM design pattern and implement it using nova-mvvm. Utilize Pydantic for data validation and defining data models for inputs.
  • Web UI Development with nova-trame: Build a web-based user interface using nova-trame and Vuetify components. Implement two-way data binding between UI elements and the ViewModel.
  • Advanced Pydantic Validation: Explore advanced Pydantic features like nested models and custom validators for more robust data validation.
  • Advanced Visualization Techniques: Integrate libraries like Plotly, PyVista and VTK with Trame to produce sophisticated 2D and 3D data visualizations.
  • Deployment to NOVA/NDIP: Learn the steps for containerizing your application with Docker, defining your tool with an XML file, and deploying it to the NOVA/NDIP platform.

Core Components and Workflow:

  1. Start with the NOVA Template: Use copier to create a starting point.
  2. Define Data with Pydantic: Create Pydantic models for structured data and validation.
  3. Implement the MVVM Pattern:
    • Use the Model layer to define the application data and business logic, including connecting to NDIP via nova-galaxy.
    • Create a ViewModel to serve as an intermediary between the Model and View, exposing data and handling user interactions.
    • Construct the View (UI) using nova-trame and Vuetify, binding UI elements to the ViewModel.
  4. Interact with NDIP: Use nova-galaxy to connect to NDIP, run tools, and manage data.
  5. Develop the User Interface: Build a web-based user interface using nova-trame and Trame.
  6. Create advanced visualizations: Integrate Plotly, PyVista, and VTK libraries to visualize data in the UI.
  7. Deploy the Application: Containerize your application and deploy it to the NDIP platform for others to use.

Outcome:

By the end of this tutorial, you will have a solid understanding of the NOVA framework, its core libraries, and the MVVM design pattern. You will be able to build interactive web applications that connect to NDIP, run neutron data analysis tools, and visualize the results, empowering you to create custom tools for the neutron scattering community. The tutorial also provides a pathway for deploying your application to the NOVA/NDIP platform.

Verify Prerequisites

  • Basic Python Knowledge: A basic understanding of Python programming concepts is required.
  • A Text Editor or IDE: You will need a text editor or IDE (such as VS Code, Sublime Text, or Atom) for writing code.
  • Familiarity with the Command Line: You will need to be comfortable using the command line or terminal.

Setup

This section guides you through setting up your development environment to follow along with the NOVA tutorial. It builds upon the prerequisites outlined earlier and provides specific instructions to ensure you have the necessary tools and libraries installed and configured.

1. Mac and Linux

  • Python Installation: You must have Python 3.11 or higher installed on your system. Verify your Python version by running python --version or python3 --version in your terminal. If python is already installed, it should be available on linux systems via your package manager. On macOS, download the installer from https://www.python.org/downloads/.
  • Pythons copier Library: We will be using this library to generate a starting application from a template. Install it using pip (ex. pip install copier).
  • Poetry: The code samples provided in this tutorial leverage Poetry for dependency management. Run the command poetry to see if you already have Poetry installed. If you dont have Poetry installed, follow the instructions on the official Poetry website: https://python-poetry.org/docs/#installation.
  • Git: You must have git installed on your system. Try running git version command in your terminal to see if it is already installed. If not, please visit the git downloads page for installation instructions. https://git-scm.com/downloads

2. Windows

For windows systems, it is recommended that you use the analysis cluster for this tutorial.

Callout

You can use the analysis cluster for this tutorial. This is recommended if you use Windows or otherwise can't meet the above prerequisites on your laptop. By default, the python command on the cluster will use 3.9, so please explicitly reference python3.11 where needed.

You can create a virtual environment suitable for the tutorial on the cluster with:

BASH 

python3.11 -m venv .venv
source .venv/bin/activate
pip install copier poetry

3. Getting your Galaxy API Key

In order to run the code examples in this tutorial, an API Key is required. An API key is obtained from the NDIP instance directly.

Callout

For this tutorial, we will be using https://calvera-test.ornl.gov as our NDIP instance. This is the instance that is used for testing and development of platform and NOVA tools. Once tools are mature and ready for users, theyll be migrated to https://calvera.ornl.gov. This process is explained in more detail at the end of the tutorial.

The steps to retrieve your API Key are:

  1. Navigate to https://calvera-test.ornl.gov
  2. Log in using your UCAMS id if necessary
  3. Under the Users dropdown menu, choose preferences
  4. Select Manage API Key
  5. You should now be on the Manage API Key page where you can view/copy the API Key.

4. Configure Environment Variables

The NOVA framework requires you to set environment variables for your NDIP URL and API key. These variables are used to authenticate with the NDIP platform.

  1. Set GALAXY_URL: Set the GALAXY_URL environment variable to the URL of your NDIP instance, in this case calvera-test.ornl.gov:

    BASH 

    export GALAXY_URL=calvera-test.ornl.gov  # Linux/macOS
set GALAXY_URL=calvera-test.ornl.gov     # Windows (Command Prompt)
$env:GALAXY_URL="calvera-test.ornl.gov"  # Windows (PowerShell)

  2. Set GALAXY_API_KEY: Set the GALAXY_API_KEY environment variable to your NDIP API key. Replace <your_api_key> with your actual API key:

    BASH 

    export GALAXY_API_KEY=<your_api_key>  # Linux/macOS
set GALAXY_API_KEY=<your_api_key>     # Windows (Command Prompt)
$env:GALAXY_API_KEY="<your_api_key>"  # Windows (PowerShell)

    Important: For security reasons, it is recommended to avoid hardcoding your API key directly in your code. Using environment variables is a more secure and flexible approach.

5. Create a GitLab Personal Access Token

Callout

If you currently use an ssh key for gitlab, you will still need to create a personal access token for the tutorial.

To interact with repositories on code.ornl.gov, youll need to create a personal access token:

  1. Navigate to https://code.ornl.gov and log in with your credentials
  2. In the left sidebar, select your avatar.
  3. Select Edit Profile
  4. On the left sidebar, select Access tokens
  5. In Token name, enter a name for the token (such as Nova Tutorial)
  6. Provide the desired scopes (at minimum, select read repository, write repository, and api)
  7. Click Create personal access token.
  8. Important Copy and save your token to your computer immediately. You will no longer have access to it after leaving the page.

6. Verify Your Setup

To ensure your setup is correct, run the following command in your terminal within the nova_tutorial directory:

BASH 

echo $GALAXY_URL
echo $GALAXY_API_KEY

This should print the values of your GALAXY_URL and GALAXY_API_KEY environment variables. If the values are printed correctly, your setup is complete.

You are now ready to start building your NOVA application! We will proceed in the next episode.