US Army ERDC Summer Internship

Responsibilities

• Researched about Nvidia DeepStream, PyTorch, and the implementation of machine learning models into the GStreamer video pipelines.
• Reported findings to mentors about the challenges and errors that can occur for video pipelines, reducing the future time and effort taken to find an optimal solution.
• Studied how to convert pretrained PyTorch models into ONNX, enabling the successful deployment of pytorch models into the video pipeline.
• Redesigned code that can train a multiclass semantic segmentation model with custom datasets, allowing SRC to deploy custom-trained segmentation video pipelines.

Background

The Supercomputing Research Center (SRC) has been researching the capabilities of machine learning models, and one such tool that utilizes these models is Nvidia’s DeepStream SDK. DeepStream SDK is a platform that allows its users to perform computer vision by sending a video through a GStreamer video pipeline which breaks down the video into frames so that it can then be passed through an inference model, producing a modified video with classifying labels. The SRC has worked with several machine learning models now, and two models that the team wanted to integrate into the video pipelines are the ESRGAN model and the MINC segmentation model.

Link to the internship experience powerpoint

Experience

This internship was an extremely valuable experience that helped improve my research and problem-skills and illustrated the traits needed to prosper in a work environment such as documentation, communication, and the perserverance to learn from many failures.

Here are some of the memorable things that I learned and did while at the internship!

• Learn about the components and function of Nvidia Deepstream SDK and GStreamer pipelines
• Gain a basic understanding of Machine Learning and PyTorch
• Navigating terminal and manipulating files using bash commands such as mv, cp, cd, and ls
• Wrote and modified code that could train binary segmentation and multiclass segmentation
• Presented my work in a professional setting
• Created a gitlab repository that contains documentation that will serve as a guideline for future projects
• Written a research paper about the internship and experience

Reflection

Initially, when I had seen the project goals that were expected for my summer internship, I was afraid of not being able to contribute anything to the project. I had seen terms I have never heard or seen of before as someone that was new to Computer Science. What was PyTorch or TensorFlow? Nvidia DeepStream, GStreamer, or HPC? I was worried that my lack of experience in the field and knowledge would lead to me not producing any meaningful contribution.

However, thankfully with the help of my mentors and staff from the team, I am able to say that my worries were proven naught. They had guided me by suggesting paths to consider for a problem or how to problem solve when something is going wrong. Most importantly, they had created an environment where I could succeed by emphasizing how it is meant to be a learning experience and not a succeed or fail project.

In turn, it led me to be more comfortable for taking initiative, to brainstorm my own possible solutions based on what I read on and seen, knowing that it was okay if it failed. It had madae me realize just how much failure happens in this field which is ok and that the best way to learn is to take small steps and to learn from the errors.

The most significant impact that this internship had on me was the experience of working in the computer science field. The knowledge and experience that I have gained while working at ERDC is one that cannot be replicated in the classroom and is something that I will take with me in future internships. Furthermore, because of this internship, it gave me the push that I needed to delve into the world of technology. Now I actively seek out to learn more about technology whether it be programming languages or improvements in technology as I have a first-hand experience of how technology is developed and researched. Because of this job, it has provided me with a better understanding of what I want to do in the future and that this field is one I can find myself working in.