From 29f48172fbc4ad8b68631efe8a4afe48b49717d7 Mon Sep 17 00:00:00 2001 From: Rob Wiederstein Date: Thu, 16 Apr 2026 06:15:07 -0400 Subject: [PATCH] Refresh data, move logs to project dir, update docs - Weekly refresh: 388 owners, 10 sales, 11 listings (2026-04-16) - Move cron logs from ~/ to logs/ in each project dir - Add logs/ to .gitignore and .dockerignore - Update CLAUDE.md with log location and ops notes - Update TODO.md with log relocation completion --- .dockerignore | 2 ++ .gitignore | 7 ++++++- CLAUDE.md | 5 +++++ TODO.md | 3 +++ data-raw/refresh_all.R | 4 ++++ data/listings.rds | Bin 640 -> 637 bytes data/sales.rds | Bin 621 -> 618 bytes 7 files changed, 20 insertions(+), 1 deletion(-) diff --git a/.dockerignore b/.dockerignore index 96f5cd2..258af7b 100644 --- a/.dockerignore +++ b/.dockerignore @@ -17,3 +17,5 @@ data-raw/ CLAUDE.md TODO.md README.md + +logs/ diff --git a/.gitignore b/.gitignore index 5197580..b656ea4 100644 --- a/.gitignore +++ b/.gitignore @@ -27,10 +27,15 @@ data-raw/uscb/ # Derived data (rebuilt from data-raw via data-raw/main.R) data/owners.rds + +# Downloaded HOA documents (large, not committed) +www/docs/ data/venice.rds data/venice_facts.rds data/beaches.rds .Renviron .aider* .env -.env + +# Logs +logs/ diff --git a/CLAUDE.md b/CLAUDE.md index 98cc653..aba0cec 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -189,3 +189,8 @@ operation with no manual steps. - Geocoding used Google API via `tidygeocoder`; results cached in `data-raw/geotagged_street_addresses.rds` to avoid re-calling the API. - Point deduplication (multiple units at same address) was done manually in QGIS — not scripted. `owners_moved.gpkg` is the authoritative geocoded dataset. - `data-raw/` is gitignored except for the shapefiles in `data-raw/PlatBoundary/` and `data-raw/SarasotaCountyBoundary/` which are committed. + +## Ops + +- **Check latest refresh log:** `tail -50 /data/projects/r/stAndrews/logs/refresh.log` +- Cron runs every Sunday at 11pm; logs go to `logs/refresh.log` (not `~/`) diff --git a/TODO.md b/TODO.md index f6a8b8b..dd6153a 100644 --- a/TODO.md +++ b/TODO.md @@ -12,6 +12,9 @@ - [x] Add cron job to run `update_owners.R` weekly inside the rstudio container: Runs every Sunday at 11pm via crontab; restarts standrews_shiny after; logs to ~/standrews_update.log Tested end-to-end 2026-03-09 — 388 owners written, options(timeout=300) required for 87.5 MB download +- [x] Move cron log from ~/ to project dir (2026-04-16): + Log now writes to /data/projects/r/stAndrews/logs/refresh.log + logs/ added to .gitignore and .dockerignore ## Features diff --git a/data-raw/refresh_all.R b/data-raw/refresh_all.R index 06e830c..45c643c 100644 --- a/data-raw/refresh_all.R +++ b/data-raw/refresh_all.R @@ -32,4 +32,8 @@ source("./data-raw/update_owners.R") cat("\n--- update_sales.R ---\n") source("./data-raw/update_sales.R") +# ── Update listings ─────────────────────────────────────────────────────────── +cat("\n--- update_listings.R ---\n") +source("./data-raw/update_listings.R") + cat("\n=== Refresh complete", format(Sys.time()), "===\n") diff --git a/data/listings.rds b/data/listings.rds index 0930be174c11c3a782168131ce2cf3d8bdecafa5..23e5cbdccbde5ffbcd8d440dbe8c8beea9d0929c 100644 GIT binary patch literal 637 zcmV-@0)qV?iwFP!000001C5lyZ_-d4#~-5;8Nvu`>cPWPV=|R3Ed`<3rMQKYUKHjI zGf>w8b*06&%i?V(qc@YysAP%%fM)8&D={(AqnU|E7w5&q#ETw`armR}Ra%v02l({+ zeZSwYzxR0j=0^}hLBudY6dZ;}G+ZtfC-dMXL4<%YU_^YESnaL$85aqo9l`_1;npv* zzV(&rPgH-Pd`112jz)+Xg0BE#P`LN&Hl;4bwe)K3X)8W&_))$6vQ z;~6`J%erb~j^$H$PSMpxC54L@FrP^aJk2?q&*kw0b!ROPzj$zi#HNvHF<+&?ej*z~{zEU;O8UNYgpZaP~ZSeVb? zi-vBShUMu+*rPC)*-v5Ertsfw?sfbp1vs0_EBmPreWdq}Lx^0>b#jr!jmLy-k?`Sb z>SOBDxb|(r2d+Qi*OYGk!l6y)iWkOCJaw47)gmNy3W-3~w|?aKHdz<^9FJ`fE_A&0 z#X9Y)M*b0Eoh*wDx(}K94eBp#6RK|fs^8bDzW&AOGo-4~Nzile~sCvklTIL91aTL!A%?pCuN zJ$n6p%mAKQiryRZXgAK)4*wnvy3|zM(Ga3$L{lwWkt=d2*; z5-k&M5EU^;R9cEz@q`@IBwMwYWj89U8G4-<{&vEqu`=bh6ot=F?^oz{yWdG*po{!d XYy^6grfMdnfZF{H0V(;Gcm)6el`cZ` literal 640 zcmV-`0)PD8dU|?WoU}0fqU}gm}8CXL@+;lB~V!}WUJCNoB(%cSL z*g%pD2%5zKLNgoy)0h7?Kvn zC$YE~$PhphWN}F>Nd?mXpyJ$6aZL+zbA^z^qLRcs1(zZnh0wgr5(Ps8BOQgX)V$2( zR2>C39|dC*BTHkb23w5wBXdC zV`*Td;Fywbi5IMbqoxU4G?j~2N1rB1B5;R zPUeoHKN%o2-yAS4rXYZ9zC#KmH#lTK?azYJIZ$_J9DvZ-51{f;er5tx-2znk+yoT& zJ6H>+{WOkMb+8uNef#l2Rv_Q}E96-gYuBXp?snFVjE?0k@UgT zfoYie2STQYu}DlV1=7W5!_7Q^v{p9nR1qNUApFLSA4v0OUFXY35{L0Kzf4FlMbZZo z2h-@qLtbKTDy(>5hYE7%WEPjCro^X!N(qP%dtypTQ7W(?ffgXeg=r-)BMORuDG18p z19RdFQj6k|lyKxEmSmQcrodEj=H%z4Lj+Kr!&#JHt_O1xFAy_`^4b)91pol3iX2q{ diff --git a/data/sales.rds b/data/sales.rds index 213a9d1ec31f73fb5affb36d37aef4b49d328982..bd0e04c3fa477da2eac99e42804d3dc0f89ecf78 100644 GIT binary patch delta 584 zcmV-O0=NC`1nLBkEq??WK(Y>3_zys6PAH!P%IARcSr{NR14R7tR|g3Fr2&~{U|?Ya z5{wKi44fcKlXDV_i-8OQBtaIJ#FA7X{SPY61r?XKFfdl|_4f-2@(*@Za0yZfbM^Cd zc2#imQ7|?!vNVP&Q#3R(QV93=cX0~xbaBV4(9n`7g+Pl>n4oGh;xzDv3NWbi2*}BicTDXy=JdPDeJD%M+?F~@8(B@@< zCXiq9@az%|AU$tVcm+Sq+yf!gnO}dFSPi5Lmz;NJ^p^%y`U6DW<+o7!A(XxaRev3;jCsUZ8jiTs@1C1p{-{0tU8b0|u6_9Skgy9t^C1O&HkpI2f4M zW-u^+U%-F+003aC0Z#w`