Rob Wiederstein
29f48172fb
Some checks failed
Deploy stAndrews / deploy (push) Failing after 16s
- Weekly refresh: 388 owners, 10 sales, 11 listings (2026-04-16) - Move cron logs from ~/ to logs/ in each project dir - Add logs/ to .gitignore and .dockerignore - Update CLAUDE.md with log location and ops notes - Update TODO.md with log relocation completion
197 lines
9.2 KiB
Markdown
197 lines
9.2 KiB
Markdown
# stAndrews — Claude Context
|
||
|
||
## Required reading
|
||
|
||
Before working on this project, read:
|
||
- `/home/rkw/.claude/CLAUDE.md` — infrastructure context (network, Docker, deployment)
|
||
- `/home/rkw/CLAUDE.md` — home directory layout
|
||
|
||
## What this project is
|
||
|
||
A Shiny mobile app (using `shinyMobile` / Framework7) that maps property owners
|
||
and community information for St. Andrews Park, Venice, Florida. 388 properties
|
||
across 14 subdivisions in Plantation Golf and Country Club.
|
||
|
||
**Entry point:** `app.R` (single-file Shiny app)
|
||
|
||
## Stack
|
||
|
||
- **UI:** `shinyMobile` (Framework7-based, iOS theme, dark mode)
|
||
- **Maps:** `leaflet` + `leafpop`
|
||
- **Tables:** `DT`
|
||
- **Spatial:** `sf`
|
||
- **Data wrangling:** `dplyr`
|
||
|
||
## Data
|
||
|
||
| File | Description | Source |
|
||
|------|-------------|--------|
|
||
| `data/owners.rds` | SF object — geocoded property owners (post-QGIS edit) | Built by `data-raw/main.R` |
|
||
| `data/plats/plats.shp` | Subdivision boundary polygons | Built by `data-raw/create_sbdv_plats.R` |
|
||
| `data/venice.rds` | Venice city boundary polygon | Built by `data-raw/create_venice_bndry.R` |
|
||
| `data/venice_facts.rds` | Venice demographic facts table | Built by `data-raw/main.R` or similar |
|
||
| `data/beaches.rds` | EPA beach monitoring locations | Built from `data-raw/epa/` |
|
||
|
||
**Raw inputs** (gitignored — large or sensitive):
|
||
- `data-raw/property/SCPA Public.xlsx` — Sarasota County Property Appraiser export
|
||
- `data-raw/addresses/owners_raw.gpkg` — geocoded points before QGIS editing
|
||
- `data-raw/addresses/owners_moved.gpkg` — points after manual QGIS adjustment (source of truth for `owners.rds`)
|
||
- `data-raw/geotagged_street_addresses.rds` — cached geocoding results (Google API)
|
||
|
||
**Rebuild pipeline:**
|
||
1. `data-raw/main.R` — geocodes addresses, exports to `owners_raw.gpkg`
|
||
2. Manual QGIS step — adjust duplicate-location points, save as `owners_moved.gpkg`
|
||
3. `data-raw/main.R` (continued) — reads QGIS output, writes `data/owners.rds`
|
||
4. `data-raw/create_sbdv_plats.R` — builds `data/plats/plats.shp`
|
||
5. `data-raw/create_venice_bndry.R` — builds `data/venice.rds`
|
||
|
||
## App tabs
|
||
|
||
| Tab | Content |
|
||
|-----|---------|
|
||
| About | Community description + photo |
|
||
| Venice | City boundary map + facts table |
|
||
| Beach | EPA beach locations map + helpful links |
|
||
| Owners | Searchable owner map + table (filter by name, address, subdivision) |
|
||
| Resources | Links to city services + PDF documents |
|
||
|
||
PDF documents are served from `www/docs/`.
|
||
|
||
## Running locally
|
||
|
||
Run inside the `rstudio` Docker container (see global CLAUDE.md):
|
||
|
||
```r
|
||
shiny::runApp("/home/rstudio/projects/r/stAndrews")
|
||
```
|
||
|
||
Or from RStudio with the project open: click **Run App**.
|
||
|
||
## Deployment
|
||
|
||
Deployed as a Shiny app. Path routing via the analytics gateway:
|
||
- Add a route to `~/docker/gateway/Caddyfile` on the analytics VM if not already present.
|
||
|
||
## Old app
|
||
|
||
Previously deployed to shinyapps.io at `https://rob-wiederstein.shinyapps.io/stAndrews/`
|
||
(account: `rob-wiederstein`, appId: 14173710). No longer maintained — app should be
|
||
archived/deleted from the shinyapps.io dashboard. The local `rsconnect/` directory
|
||
has been removed.
|
||
|
||
### Problems
|
||
|
||
**1. Geocoding was unreliable and required manual correction**
|
||
|
||
St. Andrews has three property types: standalone homes, villas (2- and 4-unit structures
|
||
where each side is a separate unit), and 8-unit buildings. The 8-unit buildings have unit
|
||
numbers (1–8) in the raw property records. Those unit numbers were stripped before
|
||
geocoding, so all 8 units in a building returned a single shared coordinate. Google's
|
||
geocoding was also only approximate — coordinates landed near addresses but not on top
|
||
of the actual structures. The result was that every point had to be manually reviewed
|
||
and dragged to its correct location in QGIS. This was labor-intensive and not repeatable.
|
||
|
||
**2. No update path**
|
||
|
||
The data pipeline was designed as a one-shot process with no way to refresh. Property
|
||
ownership changes hands regularly — the data should be updated roughly weekly. The
|
||
manual QGIS correction step makes this especially painful: any new or changed record
|
||
would require re-running geocoding and repeating the manual editing. There is no
|
||
mechanism to diff new records against existing ones or to carry forward previously
|
||
corrected coordinates.
|
||
|
||
### Geographic data flow
|
||
|
||
There are three independent geographic data flows, all ending in WGS84 (EPSG:4326):
|
||
|
||
**1. Owner points** (`data-raw/main.R`)
|
||
|
||
Raw property records from the Sarasota County Property Appraiser (Excel) are filtered
|
||
to the 13 St. Andrews subdivisions. Street addresses are parsed and assembled into
|
||
geocodable strings, then sent to the Google geocoding API via `tidygeocoder`. Results
|
||
are cached as `geotagged_street_addresses.rds` so the API isn't called twice. The
|
||
geocoded points are written to `owners_raw.gpkg`. Because many units share a building
|
||
address, they all land on the same coordinate — so the file is loaded into QGIS and
|
||
points are manually dragged to their actual locations. The adjusted file
|
||
(`owners_moved.gpkg`) is read back into R and saved as `data/owners.rds`.
|
||
|
||
**2. Subdivision boundary polygons** (`data-raw/create_sbdv_plats.R`)
|
||
|
||
A Sarasota County GIS shapefile (`data-raw/PlatBoundary/`) containing all county plat
|
||
boundaries is read in, reprojected to WGS84, and filtered to the same 13 subdivision
|
||
IDs. Subdivision names are cleaned up and the result is written to `data/plats/plats.shp`.
|
||
|
||
**3. Venice city boundary** (`data-raw/create_venice_bndry.R`)
|
||
|
||
A Sarasota County boundary shapefile (`data-raw/SarasotaCountyBoundary/`) is filtered
|
||
to the City of Venice (`municipali == "CV"`), keeping only the main polygon (acreage >
|
||
2500 to drop small outliers). It is simplified with `rmapshaper`, interior holes are
|
||
removed, reprojected to WGS84, and saved as `data/venice.rds`.
|
||
|
||
## New app — decisions
|
||
|
||
**Same folder/repo.** The UI in `app.R` is already good — tabs, maps, and layout don't
|
||
need to change. What needs replacing is entirely in `data-raw/`: the pipeline that
|
||
produces `data/owners.rds`. The app just consumes that file. Rebuild the pipeline,
|
||
keep the app.
|
||
|
||
**Feasibility hinges on the join.** The geometry side is straightforward — filter
|
||
building footprints to the subdivision boundary, compute centroids, done. The hard part
|
||
is linking those geometries to SCPA property records. Two approaches worth investigating:
|
||
|
||
- **Address join** — if the footprint layer carries situs addresses in the same format
|
||
as the SCPA records, a direct join works. If formatting differs, fuzzy matching may
|
||
be needed.
|
||
- **Account number join** — the SCPA assigns an account number to each property. It's
|
||
worth checking whether account numbers follow the *property* (stable, ideal join key)
|
||
or the *owner* (changes on sale, less useful). If account numbers are property-stable
|
||
and also appear in the footprint layer, this would be the cleanest and most reliable
|
||
join key — especially for multi-unit buildings where address disambiguation is messy.
|
||
|
||
Also needs confirming: whether the footprint polygons exist at the **unit level** or
|
||
only at the **building level**. If only building-level, the stacking problem from the
|
||
old pipeline reappears.
|
||
|
||
Data inspection will resolve all of this.
|
||
|
||
## New app — proposal
|
||
|
||
Instead of geocoding addresses, use the Sarasota County GIS building footprint layer,
|
||
which contains polygon outlines of every structure. Key advantages:
|
||
|
||
- **No geocoding API needed.** Building outlines are already precisely positioned over
|
||
actual structures. Deriving centroids from the polygons gives accurate, stable
|
||
coordinates for every unit — far better than approximate Google geocoding results.
|
||
- **Handles multi-unit buildings cleanly.** Each unit in an 8-unit building has its own
|
||
footprint polygon, so each gets its own distinct centroid. No stripping of unit numbers,
|
||
no stacking of points, no manual QGIS correction.
|
||
- **No new construction to worry about.** St. Andrews is a built-out community with no
|
||
active development, so the building footprint layer is effectively static.
|
||
|
||
**Proposed pipeline:**
|
||
|
||
1. Download the Sarasota County building footprint GIS layer.
|
||
2. Spatially filter it to the St. Andrews subdivision boundary (already have
|
||
`data/plats/plats.shp`).
|
||
3. Compute centroids for each building footprint polygon.
|
||
4. Join centroids to property records from the SCPA using address as the key.
|
||
5. Save the joined dataset as the owner points layer.
|
||
|
||
**Update path:**
|
||
|
||
Because the geometry is fixed (building centroids don't change), only the ownership
|
||
attributes need refreshing. The SCPA property records can be re-downloaded weekly and
|
||
re-joined to the stable centroid table. This makes weekly updates a simple scripted
|
||
operation with no manual steps.
|
||
|
||
## Notes
|
||
|
||
- Geocoding used Google API via `tidygeocoder`; results cached in `data-raw/geotagged_street_addresses.rds` to avoid re-calling the API.
|
||
- Point deduplication (multiple units at same address) was done manually in QGIS — not scripted. `owners_moved.gpkg` is the authoritative geocoded dataset.
|
||
- `data-raw/` is gitignored except for the shapefiles in `data-raw/PlatBoundary/` and `data-raw/SarasotaCountyBoundary/` which are committed.
|
||
|
||
## Ops
|
||
|
||
- **Check latest refresh log:** `tail -50 /data/projects/r/stAndrews/logs/refresh.log`
|
||
- Cron runs every Sunday at 11pm; logs go to `logs/refresh.log` (not `~/`)
|