Views from Intelligent Light
Roger Rintala

NASA CFD Vision 2030 Update


Scalable Knowledge Capture is Essential to Avoid CFD Bottlenecks



Image produced by Intelligent Light via XDB’s from an Air Force Research
Laboratory (AFRL) sponsored Phase II SBIR, Contract FA8650-14-C-2439.

NASA’s CFD Vision 2030 Study details the many challenges that remain to routinely obtain accurate physics-based predictions of complex turbulent flows, including how to streamline and automate analysis to gain knowledge. Evolving HPC architectures will produce huge amounts of data, and future CFD technologies must be built to both realize the promise and avoid the pitfalls of this uncertain landscape. At Aviation 2015 this summer, Intelligent Light’s Dr. Earl Duque participated in an expert panel that discussed visions for post-processing and knowledge capture to meet the NASA 2030 CFD goals. Dr. Duque will be the lead author on the summary paper targeted for SciTech 2016.


Reduced Order Modeling Identified in the Study as an Enabling Technology


Reduced  Order Modeling (ROM) can both compress and summarize, in a physics-oriented way, large unsteady CFD results and experimental data. Dr. Duque’s Applied Research Group at Intelligent Light has been successfully collaborating with BYU in an Air Force Research Laboratory-funded research effort to apply ROMs and Self-Organizing Maps (SOMs) to turbomachinery CFD. This is one example of how a partnership of government, industry and university researchers is working to make NASA’s 2030 CFD vision a reality.




Yves-Marie Lefebvre

FieldView Patch Release 15.1 Now Available


New Surface Flows seeding option, improved readers, growing pathlines and many more improvements

Surface Flows obtained with the new “From File” seeding option

Surface Flows obtained with the new “From File” seeding option


The FieldView Development Team has been working hard on great new capabilities for FieldView 16, coming this fall. We’re halfway through our yearly release cycle: the perfect time to step back and package all the new capabilities and improvements already available so you can benefit from them now.


Our recently adopted Agile development method allows us to deliver this patch release quickly so we can spend more time working on FieldView 16.  Only some of the files in your FieldView 15 installation will be updated in the process. Our next full installation package will come with FieldView 16, which will be a new major release of the type our users are more familiar with.


Highlights of this patch release are:

  • More control over the location of Surface Flow lines with the new “From File” seeding option.
  • 4x faster read times for the AcuSolve Direct Reader.
  • Parallel post-processing and support for FLOW-3D v11 with the FLOW-3D Direct Reader.
  • Animate the full path followed by your particles over time, even in moving grids, with the new “Growing” display type for particles.
  • Save time on particle animation by reading a single time step and animating only your particles with the new “Read as Steady State” option for single-file transient results.
  • Increased maximum parallel partitions count.
  • Changing grid counts in PLOT3D Function files during transient sweep.
  • A more verbose mode for debugging corrupted arbitrary polyhedra from your solver export.
  • And over 20 additional fixes and improvements…


Check out the full details in the What’s New in FieldView Patch Release 15.1 document.


The FieldView patch release 15.1 is an upgrade to your FieldView 15 installation. It will run with your current FieldView 15 passwords. It is available for download from the FieldView Customer Center.

Earl Duque

Unraveling the Mysteries of Turbulence Transport in a Wind Farm

A joint paper with Prof. Sven Schmitz was just issued in the ”Wind Turbine 2015″ special issue of the online journal Energies.


This paper entitled “Unraveling the Mysteries of Turbulence Transport in a Wind Farm” is co-authored with Pankaj K. Jha 1, Earl P. N. Duque 2, Jessica L. Bashioum 1 and Sven Schmitz 1,*


For this project, we used FieldView XDB workflows to enable the investigation of “mysteries involved in the recovery process of the wake momentum deficit, downstream of utility-scale wind turbines in the atmosphere.”  The “High-resolution surface data extracts provide new insight into the complex recovery process of the wake momentum deficit governed by turbulence transport phenomena. “

Roger Rintala

Breakthrough CFD Scalability at 64,000 cores – In Situ for Extreme Scale CFD

Intelligent Light and Georgia Tech Researchers Achieving Breakthrough CFD Scalability at 64,000 cores

Research Leading Toward Practical Extreme Scale CFD


Rutherford, NJ – June 4, 2015

Intelligent Light, in collaboration with scalable solver developers at Georgia Tech and HPC experts at Lawrence Berkeley National Laboratory, is achieving breakthrough CFD scalability running the AVF-Leslie combustion simulation code on up to 64,000 cores on supercomputers at the Department of Energy’s National Energy Research Scientific Computing Center.  The project is bringing together Intelligent Light’s expertise in computer science, software and hardware architecture, solver integration, data management, and the practical application of CFD to deliver scalable analysis methods and in situ infrastructure for extreme scale knowledge discovery.


High performance computing (HPC) is a necessity for the pursuit of high-fidelity, time-accurate simulations of sophisticated physics.  HPC makes it possible to run these highly detailed simulations and deliver results in reasonable timeframes.  When running simulations using thousands of cores however, the time to write, re-read and post-process the resulting files using traditional volume-based post-processing is impractical or impossible.  When results are not reviewed and desired simulation runs not performed due to these limitations, the cost is wasted computing resources and lost science.  In situ methods enable analysis of full spatiotemporal resolution data while it is still resident in memory, thereby avoiding the costs associated with writing very large data files to persistent storage for subsequent, post hoc analysis.


Scalable CFD analysis with extreme scale computing


For the AVF-Leslie code, a derivative of Georgia Tech’s Leslie3D solver code, breakthrough scalability is being achieved when running on up to 64,000 cores and the code has been instrumented with VisIt/Libsim to enable in situ extraction of surfaces of interest.  Previously, the code has been used for combustion simulations running up to 5,000 cores.  The extracted surfaces are output to compact XDB files for secondary processing using FieldView, Intelligent Light’s highly efficient post-processing tool that is a mainstay of CFD analysis. XDBs retain full numerical fidelity, enable both automated report generation and interactive exploration and can be used for archives.  Already, combustion researchers are simulating at this extreme scale and learning about phenomena never before possible to explore either numerically or experimentally.


“Intelligent Light has been known for years for our post-processing and visualization technology.  As the computing landscapes shifts to high-performance clusters, integrating post-processing with CFD solvers presents an opportunity to create a truly scalable workflow,” said Steve M. Legensky, General Manager and Founder at Intelligent Light.  “Although we have run the VisIt code at up to 98,000 cores on the LLNL BlueGene/Q systems, we are seeing the challenges that arise when integrating with a sophisticated physics code like AVF-Leslie.  This project, as well as others we have executed for DOE and DoD are helping us to understand the issues that affect both post-processing and solver code performance so that we can help our customers be successful in the HPC world.”


DOE taps Intelligent Light expertise in pursuit of extreme scale coherency and production quality software for real-world science


The Department of Energy (DOE) selected Intelligent Light as part of a team led by Lawrence Berkeley National Laboratory — a team that also includes Kitware, Georgia Tech and Argonne National Laboratory — to address the challenges of extreme-scale computing and integrating CFD solvers with in situ methods.  Software tools for integrating solvers with in situ methods at extreme scale must maintain coherency across tens to hundreds of thousands of processor cores and be production quality to produce useful scientific results.


“Today we see widening gaps between compute performance and I/O capability and in situ analysis is a key part of the solution. As we move toward the exascale regime, we will see 3 orders of magnitude increase in FLOPs performance while at the same time seeing only 2-3 times more I/O performance,” says Wes Bethel, Senior Computer Scientist at Lawrence Berkeley National Laboratory.  “Next generation workflows must address this discrepancy while delivering ultra-scalable performance for applications and Intelligent Light is among the organizations that are developing the proven, production quality software that will be required to produce successful science from these machines.”


DOE has assembled an exclusive team to develop the next generation methods and tools for in situ workflows to be used in a wide range of HPC-based scientific applications.  Libsim is a key interface for in situ applications.  As Libsim is tightly coupled to the solver, Intelligent Light is working with solvers to integrate this interface.  Intelligent Light is a leading developer and maintainer of the open-source VisIt application and Libsim, both developed by the DOE.


XDB – Extract database files provide essential capability for interrogation


The use of extracts permits post hoc interactive exploration using standard tools without requiring the user to know what they want to see in advance.  XDBs files are 10-1000 times smaller than solution files and are computationally efficient to create and save.  CFD users can utilize automation to analyze large volumes of data, apply new generation techniques to identify important features from across vast datasets, and maintain the ability to explore solutions interactively using FieldView – the highly efficient, user-centric post-processing product long a favorite tool of CFD practitioners across industry and research around the world.


Leading the way forward with HPC


In working on research and production projects at the extreme scale, Intelligent Light is developing leading edge expertise and experience solving the challenges that occur when in situ methods are deployed in real world applications at the extreme scale.  By understanding and solving the scalability and workflow issues at 64,000 cores and beyond, the use of HPC and in situ will be accelerated for all Intelligent Light customers.


To support the development and deployment of in situ methods, an SC15 workshop has been organized.   Research results are being periodically presented as Intelligent Light’s research and development progress.  The combustion study results were recently presented at the  27th International Conference on Parallel Computational Fluid Dynamics and presentations on related research are planned for AIAA SciTech in January, 2016.


This work is supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research under Award Number DE-SC0012449.


About Intelligent Light

Winners of multiple IDC HPC Innovation Excellence Awards, Intelligent Light provides industry-leading software and services that unlock the power and value of a highly productive CFD workflow for engineering and research organizations in a variety of industries around the world.  The company’s flagship FieldView™ product line is the most widely used CFD post-processing software for engineering and research, encompassing data management, workflow automation, visualization, and more. Intelligent Light’s expert staff provides production-related engineering services, while its Applied Research Group conducts pure research on the cutting edge of CFD science.  With customer success its paramount goal, Intelligent Light is driving real-world solutions to the toughest challenges in CFD today.


About Berkeley Lab

Lawrence Berkeley National Laboratory addresses the world’s most urgent scientific challenges by advancing sustainable energy, protecting human health, creating new materials, and revealing the origin and fate of the universe. The Berkeley Lab Computing Sciences organization provides the computing and networking resources and expertise critical to advancing the Department of Energy’s research missions: developing new energy sources, improving energy efficiency, developing new materials and increasing our understanding of ourselves, our world and our universe. Founded in 1931, Berkeley Lab’s scientific expertise has been recognized with 13 Nobel prizes. The University of California manages Berkeley Lab for the DOE’s Office of Science. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time.


Brad Whitlock

Vote of Confidence


The Department of Energy (DOE) again recognized the value of Intelligent Light’s efforts to support innovation by awarding us a Phase IIB SBIR follow-on grant to continue promising R&D on integrating FieldView and VisIt. This brings the total to $2 million that has been committed to enable FieldView to use VisIt’s scalable back end server. Bringing FieldView and VisIt together will empower FieldView users across many disciplines to gain useful insights from the largest datasets generated on the largest computers. The FieldView-VisIt integration extends FieldView’s power into the High Performance Computing (HPC) regime and brings to bear exciting technologies from VisIt such as scalable rendering. Intelligent Light’s success during the Phase II SBIR grant has translated into useful improvements to the VisIt code today and there is more to come during the Phase IIB.


Whereas prior work on FieldView-VisIt integration focused on initial coupling techniques that allow the codes to exchange data, the new work seeks to address performance of the coupling as well as the performance of VisIt itself.  In the early days of VisIt development at LLNL, we had a lot of pressure to add features as opposed to making those features work with the utmost efficiency. This means that there are a lot of places where VisIt can be sped up considerably and otherwise improved.


Related: DOE Invites Intelligent Light to Present In Situ with VisIt, Libsim, & FieldView


Performance improvements are one of the main objectives in the new work. Some of that performance will come from better utilization of parallel resources. For instance, processing an ensemble of datasets or multiple time steps can be achieved through changes in how VisIt handles the data. We plan to make changes to VisIt’s core infrastructure that enable it to process multiple datasets simultaneously in parallel so we can use more compute cores to handle a lot of intermediate sized data. These large modifications will be challenging but we know that the DOE selected Intelligent Light for our ability to carry out demanding work like this, which will benefit the larger VisIt community.


On a personal note, this will be my first time as Principal Investigator on a project of this scale. I have been a VisIt developer from the start and a figure in the VisIt community so this is a great chance for me to continue making important contributions to a code I am passionate about.