blackmarble package

blackmarble package#

Submodules#

blackmarble.download module#

class blackmarble.download.BlackMarbleDownloader(bearer: str, directory: Path)#

Bases: BaseModel

A downloader to retrieve NASA Black Marble data.

bearer#

NASA EarthData bearer token

Type

str

directory#

Local directory to which download

Type

Path

TILES: ClassVar[GeoDataFrame] = TileID                                           geometry 0    h00v00  POLYGON ((-180 90.00012, -170 90.00012, -170 8... 1    h00v01  POLYGON ((-180 80.00012, -170 80.00012, -170 7... 2    h00v02  POLYGON ((-180 70.00012, -170 70.00012, -170 6... 3    h00v03  POLYGON ((-180 60.00012, -170 60.00012, -170 5... 4    h00v04  POLYGON ((-180 50.00012, -170 50.00012, -170 4... ..      ...                                                ... 643  h35v13  POLYGON ((170.00012 -40, 180.00012 -40, 180.00... 644  h35v14  POLYGON ((170.00012 -50, 180.00012 -50, 180.00... 645  h35v15  POLYGON ((170.00012 -60, 180.00012 -60, 180.00... 646  h35v16  POLYGON ((170.00012 -70, 180.00012 -70, 180.00... 647  h35v17  POLYGON ((170.00012 -80, 180.00012 -80, 180.00...  [648 rows x 2 columns]#
URL: ClassVar[str] = 'https://ladsweb.modaps.eosdis.nasa.gov'#
bearer: str#
directory: Path#
download(gdf: GeoDataFrame, product_id: Product, date_range: List[date], skip_if_exists: bool = True)#

Downloads files asynchronously from NASA Black Marble archive.

Parameters

  • gdf (geopandas.GeoDataFrame) – Region of Interest. Converted to EPSG:4326 and intersected with Black Mable tiles

  • product (Product) – Nasa Black Marble Product Id (e.g, VNP46A1)

  • date_range (List[datetime.date]) – Date range for which to download NASA Black Marble data.

  • skip_if_exists (bool, default=True) – Whether to skip downloading data if file already exists

Returns

list – List of downloaded H5 filenames.

Return type

List[pathlib.Path]

async get_manifest(gdf: GeoDataFrame, product_id: Product, date_range: Union[date, List[date]]) → DataFrame#

Retrieve NASA Black Marble data manifest. i.d., download links.

Parameters

  • product_id (Product) – NASA Black Marble product suite (VNP46) identifier

  • date_range (datetime.date | List[datetime.date]) – Date range for which to retrieve NASA Black Marble data manifest

Returns

NASA Black Marble data manifest (i.e., downloads links)

Return type

pandas.DataFrame

model_computed_fields: ClassVar[dict[str, ComputedFieldInfo]] = {}#

A dictionary of computed field names and their corresponding ComputedFieldInfo objects.

model_config: ClassVar[ConfigDict] = {}#

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_fields: ClassVar[dict[str, FieldInfo]] = {'bearer': FieldInfo(annotation=str, required=True), 'directory': FieldInfo(annotation=Path, required=True)}#

Metadata about the fields defined on the model, mapping of field names to [FieldInfo][pydantic.fields.FieldInfo].

This replaces Model.__fields__ from Pydantic V1.

blackmarble.download.chunks(ls, n)#

Yield successive n-sized chunks from list.

async blackmarble.download.get_url(client, url, params)#
Returns

HTTP response

Return type

httpx.Response

blackmarble.extract module#

blackmarble.extract.bm_extract(gdf: GeoDataFrame, product_id: Product, date_range: Union[date, List[date]], bearer: str, aggfunc: Union[str, List[str]] = ['mean'], variable: Optional[str] = None, drop_values_by_quality_flag: List[int] = [], check_all_tiles_exist: bool = True, output_directory: Optional[Path] = None, output_skip_if_exists: bool = True)#

Extract and aggregate nighttime lights zonal statistics from NASA Black Marble.

Parameters

  • gdf (geopandas.GeoDataFrame) – Region of interest

  • product_id (Product) –

    NASA Black Marble product suite (VNP46) identifier. The available products are shown in following list:

    • VNP46A1: Daily (raw)

    • VNP46A2: Daily (corrected)

    • VNP46A3: Monthly

    • VNP46A4: Annual

  • date_range (datetime.date | List[datetime.date]) – Date range (single date or list of dates) for which to retrieve NASA Black Marble data.

  • bearer (str) – NASA Earthdata Bearer token. Please refer to the documentation.

  • aggfunc (str | List[str], default=["mean"]) – Which statistics to calculate for each zone. All possible choices are listed in rasterstats.utils.VALID_STATS.

  • variable (str, default = None) –

    Variable to create GeoTIFF raster. Further information, please see the NASA Black Marble User Guide for VNP46A1, see Table 3; for VNP46A2 see Table 6; for VNP46A3 and VNP46A4, see Table 9. By default, it uses the following default variables:

    • For VNP46A1, uses DNB_At_Sensor_Radiance_500m

    • For VNP46A2, uses Gap_Filled_DNB_BRDF-Corrected_NTL

    • For VNP46A3, uses NearNadir_Composite_Snow_Free.

    • For VNP46A4, uses NearNadir_Composite_Snow_Free.

  • drop_values_by_quality_flag (List[int], optional) –

    List of the quality flag values for which to drop data values. Each pixel has a quality flag value, where low quality values can be removed. Values are set to NA for each value in the list.

    For VNP46A1 and VNP46A2 (daily data):

    • 0: High-quality, Persistent nighttime lights

    • 1: High-quality, Ephemeral nighttime Lights

    • 2: Poor-quality, Outlier, potential cloud contamination, or other issues

    • 255: No retrieval, Fill value (masked out on ingestion)

    For VNP46A3 and VNP46A4 (monthly and annual data):

    • 0: Good-quality, The number of observations used for the composite is larger than 3

    • 1: Poor-quality, The number of observations used for the composite is less than or equal to 3

    • 2: Gap filled NTL based on historical data

    • 255: Fill value

  • check_all_tiles_exist (bool, default=True) – Check whether all Black Marble nighttime light tiles exist for the region of interest. Sometimes not all tiles are available, so the full region of interest may not be covered. By default (True), it skips cases where not all tiles are available.

  • output_directory (pathlib.Path, optional) – Directory to produce output. By default, the output will be produced onto a temporary directory.

  • outout_skip_if_exists (bool, default=True) –

    Whether to skip downloading or extracting data if the data file for that date already exists.

    bearer

Returns

Zonal statistics dataframe

Return type

pandas.DataFrame

blackmarble.raster module#

blackmarble.raster.bm_raster(gdf: GeoDataFrame, product_id: Product, date_range: Union[date, List[date]], bearer: str, variable: Optional[str] = None, drop_values_by_quality_flag: List[int] = [], check_all_tiles_exist: bool = True, output_directory: Optional[Path] = None, output_skip_if_exists: bool = True)#

Create a stack of nighttime lights rasters by retrieiving from NASA Black Marble data.

Parameters

  • roi (geopandas.GeoDataFrame) – Region of interest

  • product_id (Product) –

    NASA Black Marble product suite (VNP46) identifier. The available products are shown in following list:

    • VNP46A1: Daily (raw)

    • VNP46A2: Daily (corrected)

    • VNP46A3: Monthly

    • VNP46A4: Annual

  • date_range (datetime.date | List[datetime.date]) – Date range (single date or list of dates) for which to retrieve NASA Black Marble data.

  • bearer (str) –

    NASA Earthdata Bearer token. Please refer to the documentation.

  • variable (str, default = None) –

    Variable for which to a GeoTIFF raster. Further information, please see the NASA Black Marble User Guide for VNP46A1, see Table 3; for VNP46A2 see Table 6; for VNP46A3 and VNP46A4, see Table 9. By default, it uses the following default variables:

    • For VNP46A1, uses DNB_At_Sensor_Radiance_500m

    • For VNP46A2, uses Gap_Filled_DNB_BRDF-Corrected_NTL

    • For VNP46A3, uses NearNadir_Composite_Snow_Free.

    • For VNP46A4, uses NearNadir_Composite_Snow_Free.

  • drop_values_by_quality_flag (List[int], optional) –

    List of the quality flag values for which to drop data values. Each pixel has a quality flag value, where low quality values can be removed. Values are set to NA for each value in the list.

    For VNP46A1 and VNP46A2 (daily data):

    • 0: High-quality, Persistent nighttime lights

    • 1: High-quality, Ephemeral nighttime Lights

    • 2: Poor-quality, Outlier, potential cloud contamination, or other issues

    • 255: No retrieval, Fill value (masked out on ingestion)

    For VNP46A3 and VNP46A4 (monthly and annual data):

    • 0: Good-quality, The number of observations used for the composite is larger than 3

    • 1: Poor-quality, The number of observations used for the composite is less than or equal to 3

    • 2: Gap filled NTL based on historical data

    • 255: Fill value

  • check_all_tiles_exist (bool, default=True) – Check whether all Black Marble nighttime light tiles exist for the region of interest. Sometimes not all tiles are available, so the full region of interest may not be covered. By default (True), it skips cases where not all tiles are available.

  • output_directory (pathlib.Path, optional) – Directory to produce output. By default, the output will be produced onto a temporary directory.

  • output_skip_if_exists (bool, default=True) – Whether to skip downloading or extracting data if the data file for that date already exists.

Returns

xarray.Dataset containing a stack of nighttime lights rasters

Return type

xarray.Dataset

blackmarble.raster.h5_to_geotiff(f: Path, /, variable: str = None, drop_values_by_quality_flag: List[int] = [255], output_directory: Path = None)#

Convert HDF5 file to GeoTIFF for a selected (or default) variable from NASA Black Marble data

Parameters

  • f (Path) – H5DF filename

  • variable (str, optional) –

    Variable for which to create a GeoTIFF raster. If None, defaults will be used based on the product.

    • VNP46A1: ‘DNB_At_Sensor_Radiance_500m’

    • VNP46A2: ‘Gap_Filled_DNB_BRDF-Corrected_NTL’

    • VNP46A3: ‘NearNadir_Composite_Snow_Free’

    • VNP46A4: ‘NearNadir_Composite_Snow_Free’

    Further information, please see the NASA Black Marble User Guide for VNP46A1, see Table 3; for VNP46A2 see Table 6; for VNP46A3 and VNP46A4, see Table 9.

  • drop_values_by_quality_flag (List[int], optional) – List of the quality flag values for which to drop data values. Each pixel has a quality flag value, where low quality values can be removed. Values are set to NA for each value in the list.

  • output_directory (Path, optional) – Directory to save the output GeoTIFF file. If None, uses the same directory as input file.

Returns

output_path – Path to the exported GeoTIFF raster.

Return type

Path

blackmarble.raster.transform(da: DataArray)#

Return Affice transformation

blackmarble.types module#

Types for blackmarblepy

class blackmarble.types.Product(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)#

Bases: Enum

NASA Black Marble product suite (VNP46)

VNP46A1 = 'VNP46A1'#
VNP46A2 = 'VNP46A2'#
VNP46A3 = 'VNP46A3'#
VNP46A4 = 'VNP46A4'#

Module contents#

Contents