ORACLES RSP Neural Network cloud retrieval data. POC: Kirk Knobelspiesse kirk.knobelspiesse@nasa.gov Michal Segal Rozenhaimer Dan Miller Updated as of 2018/05/07 Main level components ==================================================================================== README.TXT processed raw trainingsets ./processed: Neural Network applied to observations ====================================================== ORACLES_2016_results ./processed/ORACLES_2016_results: stnd_ref_i_stnd_dolp_csvs stnd_ref_i_stnd_dolp_figs ./processed/ORACLES_2016_results/stnd_ref_i_stnd_dolp_csvs: data in CSV format RSP_NN_cloud_prediction_stnd_ref_i_stnd_dolp_20160910.csv RSP_NN_cloud_prediction_stnd_ref_i_stnd_dolp_20160918.csv RSP_NN_cloud_prediction_stnd_ref_i_stnd_dolp_20160925.csv RSP_NN_cloud_prediction_stnd_ref_i_stnd_dolp_20160912.csv RSP_NN_cloud_prediction_stnd_ref_i_stnd_dolp_20160920.csv RSP_NN_cloud_prediction_stnd_ref_i_stnd_dolp_20160927.csv RSP_NN_cloud_prediction_stnd_ref_i_stnd_dolp_20160914.csv RSP_NN_cloud_prediction_stnd_ref_i_stnd_dolp_20160922.csv RSP_NN_cloud_prediction_stnd_ref_i_stnd_dolp_dataOriginal.csv RSP_NN_cloud_prediction_stnd_ref_i_stnd_dolp_20160916.csv RSP_NN_cloud_prediction_stnd_ref_i_stnd_dolp_20160924.csv RSP_NN_cloud_prediction_stnd_ref_i_stnd_dolp_dataSmooth.csv ./processed/ORACLES_2016_results/stnd_ref_i_stnd_dolp_figs: automatically generated figures NN_RSP_20160910_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_original.png NN_RSP_20160920_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_original.png NN_RSP_20160910_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_smooth.png NN_RSP_20160920_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_smooth.png NN_RSP_20160912_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_original.png NN_RSP_20160922_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_original.png NN_RSP_20160912_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_smooth.png NN_RSP_20160922_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_smooth.png NN_RSP_20160914_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_original.png NN_RSP_20160924_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_original.png NN_RSP_20160914_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_smooth.png NN_RSP_20160924_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_smooth.png NN_RSP_20160916_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_original.png NN_RSP_20160925_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_original.png NN_RSP_20160916_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_smooth.png NN_RSP_20160925_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_smooth.png NN_RSP_20160918_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_original.png NN_RSP_20160927_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_original.png NN_RSP_20160918_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_smooth.png NN_RSP_20160927_ER-2_reff_veff_cot_timeseries_orig_stnd_ref_i_stnd_dolp_smooth.png ./raw: RSP data processed and ready for ingest into NN algorithm ======================================== NN_ORACLES_2016er2.tar.gz -> Processed ORACLES RSP files, 2016/ER-2. Screened using CLDV1 product for valid retrievals NN_ORACLES_2016er2_all.tar.gz -> Processed ORACLES RSP files, 2016/ER-2. Unscreened, but CLDv1 product flags included NN_ORACLES_2017p3.tar.gz -> Processed ORACLES RSP files, 2017/P-3. Screened using CLDV1 product for valid retrievals NN_ORACLES_2017p3_all.tar.gz -> Processed ORACLES RSP files, 2017/P-3. Unscreened, but CLDv1 product flags included ./trainingsets: Used to create Neural Network (note _std files are latest version) ======================= NN_20170505_ER2 -> ER-2 training set NN_20170627-20170703_P3 -> P-3 training set (complete, use this one) NN_20170627_P3 -> P-3 training set (old with fewer simulated cloud top heights) ./trainingsets/NN_20170505_ER2: NN_clouds_20170505_PP.nc NN_clouds_20170505_sampled_cut.nc README.txt NN_clouds_20170505_PP_cut.nc NN_clouds_20170505_sampled_cut_std.nc rho70_cut.zip NN_clouds_20170505_sampled.nc NN_withnoise.tar.gz rho90_cut.zip Training set grid for 2016/ER-2 Cloud Top Height (m) [1000] (assumes 19,000m aircraft altitude) (1) Cloud Optical Depth [2.5, 5.0, 10.0, 15.0, 20.0, 30.0] (6) Effective Radius (µm) [5.0, 7.5, 10.0, 12.5, 15.0, 20.0] (6) Effective Variance [0.01, 0.03, 0.05, 0.07, 0.1, 0.15] (6) Solar Zenith Angle [10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65] (12) Relative Azimuth Angle [0, 2, 4, 5, 8, 12, 16, 20, 24, 28, 32, 40, 50, 60, 70, 80, 90] (17) Total 44,064 cases ./trainingsets/NN_20170627-20170703_P3: NN_clouds_20170627-20170703.nc.gz NN_clouds_20170627-20170703_std.nc README.txt Training set grid for 2017/P-3 Cloud Top Height (m) [1000] (1) Aircraft altitude (m) [5000, 6000, 7000] (3) Cloud Optical Depth [2.5, 5.0, 10.0, 15.0, 20.0, 30.0] (6) Effective Radius (µm) [5.0, 7.5, 10.0, 12.5, 15.0, 20.0] (6) Effective Variance [0.01, 0.03, 0.05, 0.07, 0.1, 0.15] (6) Solar Zenith Angle [5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65] (13) Relative Azimuth Angle [0 to 90 in increments of 3] (31) Total 261,144 cases ./trainingsets/NN_20170627_P3: NN_clouds_20170627.nc.gz