Jan Mandel/Blog/2010 Aug Sep
August 4, 2010
Loren Cobb: Statistical estimation of covariance by spectral analysis
- Wikipedia:Spectral density estimation
- Wikipedia:Stationary process
- Wikipedia:Ergodic theory
- Spectral Analysis and Its Applications, Gwilym M. Jenkins and Donald Watts, ISBN 978-1892803030 Google books
- Board Photos
- Real (Jon): getting nans even on coarsest domain, trying stock 3.2 & wrfhelp, alternatively: use print_2d_stats, print_3d_stats, they check for nans. branch jb/mc. Meadow Creek fire http://www.geomac.gov
- Ignition interpolation (Volodymyr) branch vk/test
- define lfn(x,y) = a*c+b*y+c, tign(x,y) = time_now - d*lfn(x,y)
- on one cell, compare with numerical quadrature, error should go to 0 as quadrature submesh is refined
- on the whole domain, the result should be independent of the mesh size m n
- Visualization (Bedrich): convert fire by nc2vdf, atm by wrf2vdf, first make files smaller by ncks, see if it transfers attributes correctly. See openwfm:User:Jmandel/WRF-Fire to VAPOR conversion. But I owe him How to interpret WRF-Fire variables,
it is still a stub... done. and fix up the matlab codes so that they actually work per How to visualize WRF-Fire output in Matlab... done.
- Wind (Jan): I was trying to be too smart when writing the interpolation originally and now it's biting me... branch jm2/wind_h done.
- Adam is continuing on branch fireflux
August 11, 2010
- Loren Cobb: review of Ashok Krishnamurthy, Loren Cobb, Jan Mandel, and Jonathan D. Beezley, Bayesian Tracking of Emerging Epidemics Using Optimal Statistical Interpolation, 2010 Joint Statistical Meetings, Vancouver, Canada, July 31- August 5, 2010 pdf pptx
- Jan Mandel: Matrix structure in EnKF and crosscovariances. Board photos
August 17, 2010
August 18, 2010
Data Assimilation seminar: Abhijit Sarkar
- Abhijit Sarkar, Stochastic PDEs, domain decomposition, and filtering
- A scalable domain domain decomposition solver for stochastic PDEs (i.e. a two level preconditioner with a coarse grid).
- Data assimilation (nonlinear filtering and some full Bayesian approaches)
- Nonlinear (aeroelastic) oscillators (ODEs exhibiting Hopf bifurcation) describing the flutter of aircraft wings. We looked at parameter estimation and combined state and parameter estimation problems from Wind Tunnel Test data.
- Nonlinear Shallow water equations (depth average Navier Stokes equation) for forecasting floods when a dam breaks and Advection diffusion equation for contaminant tracking. State estimation problem using Ensemble Kalman filter.
- Long term interest is on how to couple the domain decomposition solver with the sequential data assimilation method
- Sarkar, Benabbou, Ghanem, Domain decomposition of stochastic PDEs: theoretical formulations, IJNME 2009
August 20, 2010
- VAPOR visualization: need all variables on same mesh, cannot do finer 2D fire. Cannot smooth blocky fire either. Need to process jpg files to compressed video on Mac. Will also look at VisIt. See also this List of visualization programs
- How to interpret WRF-Fire variables
- Fuel integration standalone test OK, time to use for real and merge with master. Merge procedure: 1. merge master into vk/test 2. after testing merge into master. Testing: inspection for reasonable results. Parallel test: configure -d, build DM+SM, test all 4 (serial, SM, DM, DM+SM) on various number of processors, compare wrfout/wrfrst files by loading into matlab (any of ncload,ncdump,ncread,nc2struct), must be bit-identical.
- Wind interpolation to height branch jm2/wind2 serial OK but parallel test still does not give identical results.
- Real run: problem with nesting, large winds at child domain boundary in presence of large terrain gradients. Reported by others before.
August 23, 2010
Data Assimilation seminar
- Data assimilation with nongaussian perturbations
- Elliptical distributions, generating functions,...
August 30, 2010
- Data assimilation with nongaussian perturbations II -discussion
- Assumes joint distribution of prior and data error is elliptical and uncorrelated. The posterior is then also elliptical. Multiple cycles do not reduce to 2 term recursion as they do for gaussian (Kalman filter).
September 1, 2010
- Fuel: omit layer next to boundary because tign not known outside. Use max(ids+1,its), etc., not if inside loop. Need to run the global invariance test on a subdmain only.
- Wind: vertical interpolation OK, WRF after few thousand steps gives different numbers?? Detected by parallel xor checksum (subroutine print_chsum). Now in master, but standalone broken because z0 is not in wrfinput.
- Ignition: Fire spin up, time by ESMF API
- Submitted abstracts for the AGU meeting: Adam, Jon, myself. Asked AGU, no objections to posting. Search all submitted abstracts here until September 9.
- VAPOR visualization of fireflux_med simulation with surface image from Google Earth.
September 8, 2010
- Four Mile Canyon fire in progress, emerging standard: data mostly available in Google Earth format
- API for ignition by level set function history coming
- Geolocation of fireflux image: adjust by hand
- Fuel: essentially done, need to describe testing
September 13, 2010
- Ashok Krishnamurthy, Optimal Statistical Interpolation pdf
- Google Earth. Google API seems visualization standard both for wildfire and epidemic now. Output KML files... routine for surface data, almost all from the Four Mile Canyon fire was available as a Google Earth file. Can we have weather & fire as 3D COLLADA file? (file format used also for games such as Unreal engine). Check out Google Earth API for web-based display!
September 15, 2010
- Standalone keeps breaking with natural evolution of the main code....
- Levelset history API for ignition done
- Use ESMF API to find seconds from simulation start
- Will need geolocation of the fire grid in code using the API (fxlong, fxlat) in wrfinput_d01. Jon will add in API and in real, Jan in ideal.
- Meadow creek fire atmosphere instabilities/wiggles at lateral boundaries esp. the top, is that normal? See files in /storage/jbeezley/wrfv2_fire.test/em_real/ptop . Document, ask Adam and wrfhelp.
- Archive Four Mile Canyon fire data with links (Jan) - MODIS, perimeters, infrared,...
- Got new info about and references for 20ft wind from Adam, Baughman and Albini 1980, checking out the title in Google found BehavePlus publications
- Tried to visualize 20ft wind with ROS, run in Matlab problems - fireflux_med wrfout is 60GB, running on frontend choked on rendering over X... VNC is better but the X desktop stinks from a Mac.
- Will keep trying in Matlab, identify right file and pass to Bedrich for Vapor.
September 20, 2010
- Loren's presentation to the Regents of the National Library of Medicine
- Power spectrum estimation: the reason for smoothing (such as, a frequency window) is to grow the number of samples (here, the period, by ergodicity) while keeping the estimated quantity (sum of the squares of the Fourier coefficients over the frequencies in the window) the same.... the power spectrum estimator itself does not converge... in other words, the power spectrum converges weakly but not pointwise in any sense....
September 22, 2010
- Ignition from level set function done - but needed only one 2D array with ignition time and one number, the time when the model takes over
- FireFlux visualizations: of interest
- pressure-volume visualization, bubble anywhere?
- wind going through or around the firefront?
- WPS - done fxlong fxlat for ignition location.
- Google Earth - Chris' student: Jon may write a script to get her started- mapping an image onto the Earth surface is easy given coordinates. The processing of model output is basically just to create that image - read netcdf, write image (tiff, geotiff?) Google Earth Pro not needed. Given huge datasets (TB easily) needs to work remotely from command line. The real work to do: auto-updating kml for a real-time forecast, animation, streaming, 3D objects - flames, is 3D atmosphere visualization possible yet?
- Fuel integration: needs few more days
September 27, 2010
- collected some links about the Fourmile Canyon fire
- Jon figured out useful things about Visualization in Google Earth
- KML files might contain only links, not data - need to check for archiving
September 29, 2010
- Volodymyr finished fuel integration, will continue on cleanup
- Received Google Earth code from Craig
- Visualization: show negative pressure bubble if present, compare with drawings and analytic arguments in Achtemeier-2007-RFS. See if can get pretty pictures with flow tubes, clear flow features, etc., similarly as in Norton-2009-AVH. Maybe show pressure on several horizontal sections, add wind arrows on the sections, add some flow tubes throughout, and some vertical profiles - can twist a wind speed graph around a pole, like the meteo towers in field, maybe like this ?
- The time has come to understand WRF equations, how did they actually get their form of Euler equations ? The variables are partly from Laprise-1992-EEM. Maybe some early WRF papers have more details.
- For Monday's CCM colloquium: FireFlux surface movie in Google Earth from Jon
- Ignition: Replaying ignition times with the level set PDE off done, no change in the week. Hung on walking ignition, how to store and replay ignition times when the PDE needs to advance. Maybe just some small change to the existing code?