Summary and Schedule

Neutrons Open Visualization and Analysis Framework Tutorial

This tutorial guides you through the process of building applications for neutron data analysis using the NOVA framework. NOVA provides key components to create applications that range from simple scripts, all the way to full user interfaces with advanced visualization capabilities. It leverages a Model-View-ViewModel (MVVM) architectural pattern and three key NOVA libraries: nova-galaxy, nova-trame, and nova-mvvm, to simplify interaction with the Neutron Data Interpretation Platform (NDIP).

Key Learning Objectives:

• Introduction to NDIP and NOVA: Understand the purpose of NDIP as a workflow management system for neutron data and NOVA as a framework for simplifying NDIP application development.
• Project Setup: Learn how to use the copier library to clone a template application, providing a basic project structure and pre-configured files.
• NDIP Interaction with nova-galaxy: Master the use of nova-galaxy to connect to NDIP, define and run tools, and manage input parameters.
• MVVM Implementation with nova-mvvm and Pydantic: Grasp the principles of the MVVM design pattern and implement it using nova-mvvm. Utilize Pydantic for data validation and defining data models for inputs.
• Web UI Development with nova-trame: Build a web-based user interface using nova-trame and Vuetify components. Implement two-way data binding between UI elements and the ViewModel.
• Advanced Pydantic Validation: Explore advanced Pydantic features like nested models and custom validators for more robust data validation.
• Advanced Visualization Techniques: Integrate libraries like Plotly, PyVista and VTK with Trame to produce sophisticated 2D and 3D data visualizations.
• Deployment to NOVA/NDIP: Learn the steps for containerizing your application with Docker, defining your tool with an XML file, and deploying it to the NOVA/NDIP platform.

Core Components and Workflow:

1. Start with the NOVA Template: Use copier to create a starting point.
2. Define Data with Pydantic: Create Pydantic models for structured data and validation.
3. Implement the MVVM Pattern:
   • Use the Model layer to define the application data and business logic, including connecting to NDIP via nova-galaxy.
   • Create a ViewModel to serve as an intermediary between the Model and View, exposing data and handling user interactions.
   • Construct the View (UI) using nova-trame and Vuetify, binding UI elements to the ViewModel.
4. Interact with NDIP: Use nova-galaxy to connect to NDIP, run tools, and manage data.
5. Develop the User Interface: Build a web-based user interface using nova-trame and Trame.
6. Create advanced visualizations: Integrate Plotly, PyVista, and VTK libraries to visualize data in the UI.
7. Deploy the Application: Containerize your application and deploy it to the NDIP platform for others to use.

Outcome:

By the end of this tutorial, you will have a solid understanding of the NOVA framework, its core libraries, and the MVVM design pattern. You will be able to build interactive web applications that connect to NDIP, run neutron data analysis tools, and visualize the results, empowering you to create custom tools for the neutron scattering community. The tutorial also provides a pathway for deploying your application to the NOVA/NDIP platform.

	Setup Instructions	Download files required for the lesson
Duration: 00h 00m	1. Introduction to NOVA and NDIP	What is the Neutron Data Interpretation Platform (NDIP)? What is NOVA, and how does it simplify NDIP application development? What are the key components of NOVA, and what problems do they solve? How do NOVA libraries interact with the NDIP platform? What will I be able to do after completing this tutorial?
Duration: 00h 20m	2. Getting Started with a Template Application	How do I quickly set up a starting point for a NOVA project? What files and directories are included in the NOVA template application? How does poetry manage project dependencies and virtual environments?
Duration: 00h 38m	3. Programming with NDIP	How can I interact with the NDIP platform programmatically from Python? What is the nova-galaxy library, and how does it simplify NDIP operations? How do I define an NDIP tool and specify its input parameters using nova-galaxy? Where can I find information about what NDIP tool to use and parameters to set?
Duration: 01h 26m	4. User Interface Best Practices: The MVVM Design Pattern	What is the Model-View-ViewModel (MVVM) design pattern, and why is it useful for UI development? What are the roles and responsibilities of the Model, View, and ViewModel in the MVVM pattern? How does data binding work in MVVM, and why is it important? How does the nova-mvvm library simplify the implementation of the MVVM pattern in NOVA applications? What is Pydantic, and how can it be used for data modeling and validation in the context of MVVM?
Duration: 02h 13m	5. Web-based User Interface Development	What is Trame, and why should I use it for building UIs in NOVA? How does nova-trame make Trame development easier? What are the key advantages of using a declarative UI approach with Trame? How can I create a basic UI layout using nova-trame components? How can I add common Vuetify components (e.g., VTextField, VCheckbox, VSlider) to my Trame application? How can I customize the appearance of Vuetify components? Where can I find more information about available Vuetify components?
Duration: 02h 51m	6. Advanced Data Validation with Pydantic	Why is data validation important? What is Pydantic and how it works? How can I represent complex data structures with nested relationships using Pydantic? How can I enforce validation rules that go beyond basic type checking using Pydantic? How do I use Pydantic models in NOVA framework?
Duration: 03h 51m	7. Advanced Visualizations	How can I create interactive 2D charts in my NOVA application? How can I integrate Plotly charts into Trame applications? How can I create interactive 3D visualizations in my NOVA application? What are the advantages and disadvantages of using PyVista vs.VTK for 3D visualizations? How can I integrate PyVista visualizations into Trame applications? How can I work directly with VTK for more advanced 3D visualizations in Trame? What are the key components of a VTK rendering pipeline?
Duration: 04h 52m	8. Development Cycle and Next Steps
Duration: 05h 12m	Finish

The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.

Verify Prerequisites

• Basic Python Knowledge: A basic understanding of Python programming concepts is required.
• A Text Editor or IDE: You will need a text editor or IDE (such as VS Code, Sublime Text, or Atom) for writing code.
• Familiarity with the Command Line: You will need to be comfortable using the command line or terminal.

Setup

This section guides you through setting up your development environment to follow along with the NOVA tutorial. It builds upon the prerequisites outlined earlier and provides specific instructions to ensure you have the necessary tools and libraries installed and configured.

1. Mac and Linux

• Python Installation: You must have Python 3.11 or higher installed on your system. Verify your Python version by running python --version or python3 --version in your terminal. If python is already installed, it should be available on linux systems via your package manager. On macOS, download the installer from https://www.python.org/downloads/.
• Pythons copier Library: We will be using this library to generate a starting application from a template. Install it using pip (ex. pip install copier).
• Poetry: The code samples provided in this tutorial leverage Poetry for dependency management. Run the command poetry to see if you already have Poetry installed. If you dont have Poetry installed, follow the instructions on the official Poetry website: https://python-poetry.org/docs/#installation.
• Git: You must have git installed on your system. Try running git version command in your terminal to see if it is already installed. If not, please visit the git downloads page for installation instructions. https://git-scm.com/downloads

2. Windows

For windows systems, it is recommended that you use the analysis cluster for this tutorial.

Callout

You can use the analysis cluster for this tutorial. This is recommended if you use Windows or otherwise can't meet the above prerequisites on your laptop. By default, the python command on the cluster will use 3.9, so please explicitly reference python3.11 where needed.

You can create a virtual environment suitable for the tutorial on the cluster with:

BASH

python3.11 -m venv .venv
source .venv/bin/activate
pip install copier poetry

3. Getting your Galaxy API Key

In order to run the code examples in this tutorial, an API Key is required. An API key is obtained from the NDIP instance directly.

Callout

For this tutorial, we will be using https://calvera-test.ornl.gov as our NDIP instance. This is the instance that is used for testing and development of platform and NOVA tools. Once tools are mature and ready for users, theyll be migrated to https://calvera.ornl.gov. This process is explained in more detail at the end of the tutorial.

The steps to retrieve your API Key are:

1. Navigate to https://calvera-test.ornl.gov
2. Log in using your UCAMS id if necessary
3. Under the Users dropdown menu, choose preferences
4. Select Manage API Key
5. You should now be on the Manage API Key page where you can view/copy the API Key.

4. Configure Environment Variables

The NOVA framework requires you to set environment variables for your NDIP URL and API key. These variables are used to authenticate with the NDIP platform.

1. Set GALAXY_URL: Set the GALAXY_URL environment variable to the URL of your NDIP instance, in this case calvera-test.ornl.gov:

   BASH

   export GALAXY_URL=calvera-test.ornl.gov  # Linux/macOS
   set GALAXY_URL=calvera-test.ornl.gov     # Windows (Command Prompt)
   $env:GALAXY_URL="calvera-test.ornl.gov"  # Windows (PowerShell)

2. Set GALAXY_API_KEY: Set the GALAXY_API_KEY environment variable to your NDIP API key. Replace <your_api_key> with your actual API key:

   BASH

   export GALAXY_API_KEY=<your_api_key>  # Linux/macOS
   set GALAXY_API_KEY=<your_api_key>     # Windows (Command Prompt)
   $env:GALAXY_API_KEY="<your_api_key>"  # Windows (PowerShell)

   Important: For security reasons, it is recommended to avoid hardcoding your API key directly in your code. Using environment variables is a more secure and flexible approach.

5. Create a GitLab Personal Access Token

Callout

If you currently use an ssh key for gitlab, you will still need to create a personal access token for the tutorial.

To interact with repositories on code.ornl.gov, youll need to create a personal access token:

1. Navigate to https://code.ornl.gov and log in with your credentials
2. In the left sidebar, select your avatar.
3. Select Edit Profile
4. On the left sidebar, select Access tokens
5. In Token name, enter a name for the token (such as Nova Tutorial)
6. Provide the desired scopes (at minimum, select read repository, write repository, and api)
7. Click Create personal access token.
8. Important Copy and save your token to your computer immediately. You will no longer have access to it after leaving the page.

6. Verify Your Setup

To ensure your setup is correct, run the following command in your terminal within the nova_tutorial directory:

BASH

echo $GALAXY_URL
echo $GALAXY_API_KEY

This should print the values of your GALAXY_URL and GALAXY_API_KEY environment variables. If the values are printed correctly, your setup is complete.

You are now ready to start building your NOVA application! We will proceed in the next episode.