ENH: cache downloaded atmosphere netCDF datasets (#654) - #1137
ENH: cache downloaded atmosphere netCDF datasets (#654)#1137thatrandomasiandev wants to merge 2 commits into
Conversation
Collapse the cache-key assignment the formatter wants on one line. No behaviour change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Not ready to merge yet. Three items, the third being the one I care most about. 1. Three of this PR's own new tests fail on all six platforms. 2. pylint fails (exit 8): Please split those into smaller functions rather than adding 3. Cache invalidation — the design point I want resolved before this lands. The cache is enabled by default, and the key contains neither a TTL nor the forecast model run/cycle. So a user who runs a GFS forecast today and again tomorrow silently gets yesterday's forecast, with no indication anything is stale. For forecast products that is a wrong-answer bug, not a stale-file annoyance, and it is the kind of thing that is very hard for a user to notice. What I would want:
Happy to revisit once those are addressed. |
Summary
.ncfiles under~/.rocketpy_cache/atmosphere(override withROCKETPY_CACHE).set_atmospheric_modelcalls reuse disk.no_cache=FalsetoEnvironment.set_atmospheric_model(and Windy processing) so callers can force a fresh download without breaking existing callers.Notes / TODOs
OPeNDAP "Best" aggregations are virtual catalogs, not downloadable files, so this PR caches the extracted profiles RocketPy already materializes after the first fetch (same end-user outcome: reruns skip the network). Remaining follow-ups:
.nc(non-OPeNDAP)Fixes #654
Test plan
PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 pytest tests/unit/environment/test_atmosphere_cache.pyno_cache=Trueforces re-download