Add deployment pipeline and clean up repo
- Add Dockerfile, docker-compose.yml, .dockerignore, .env (port 3842) - Add Caddyfile.snippet for analytics gateway import pattern - Add .gitea/workflows/deploy.yaml for act_runner SSH deploy - Untrack sensitive/data files (SCPA xlsx, owners.rds) - Add renv lockfile and infrastructure files - Reorganize data-raw scripts and add SarasotaCounty boundary data - Move www assets to www/images/, add docs PDFs
This commit is contained in:
191
CLAUDE.md
Normal file
191
CLAUDE.md
Normal file
@@ -0,0 +1,191 @@
|
||||
# stAndrews — Claude Context
|
||||
|
||||
## Required reading
|
||||
|
||||
Before working on this project, read:
|
||||
- `/home/rkw/.claude/CLAUDE.md` — infrastructure context (network, Docker, deployment)
|
||||
- `/home/rkw/CLAUDE.md` — home directory layout
|
||||
|
||||
## What this project is
|
||||
|
||||
A Shiny mobile app (using `shinyMobile` / Framework7) that maps property owners
|
||||
and community information for St. Andrews Park, Venice, Florida. 388 properties
|
||||
across 14 subdivisions in Plantation Golf and Country Club.
|
||||
|
||||
**Entry point:** `app.R` (single-file Shiny app)
|
||||
|
||||
## Stack
|
||||
|
||||
- **UI:** `shinyMobile` (Framework7-based, iOS theme, dark mode)
|
||||
- **Maps:** `leaflet` + `leafpop`
|
||||
- **Tables:** `DT`
|
||||
- **Spatial:** `sf`
|
||||
- **Data wrangling:** `dplyr`
|
||||
|
||||
## Data
|
||||
|
||||
| File | Description | Source |
|
||||
|------|-------------|--------|
|
||||
| `data/owners.rds` | SF object — geocoded property owners (post-QGIS edit) | Built by `data-raw/main.R` |
|
||||
| `data/plats/plats.shp` | Subdivision boundary polygons | Built by `data-raw/create_sbdv_plats.R` |
|
||||
| `data/venice.rds` | Venice city boundary polygon | Built by `data-raw/create_venice_bndry.R` |
|
||||
| `data/venice_facts.rds` | Venice demographic facts table | Built by `data-raw/main.R` or similar |
|
||||
| `data/beaches.rds` | EPA beach monitoring locations | Built from `data-raw/epa/` |
|
||||
|
||||
**Raw inputs** (gitignored — large or sensitive):
|
||||
- `data-raw/property/SCPA Public.xlsx` — Sarasota County Property Appraiser export
|
||||
- `data-raw/addresses/owners_raw.gpkg` — geocoded points before QGIS editing
|
||||
- `data-raw/addresses/owners_moved.gpkg` — points after manual QGIS adjustment (source of truth for `owners.rds`)
|
||||
- `data-raw/geotagged_street_addresses.rds` — cached geocoding results (Google API)
|
||||
|
||||
**Rebuild pipeline:**
|
||||
1. `data-raw/main.R` — geocodes addresses, exports to `owners_raw.gpkg`
|
||||
2. Manual QGIS step — adjust duplicate-location points, save as `owners_moved.gpkg`
|
||||
3. `data-raw/main.R` (continued) — reads QGIS output, writes `data/owners.rds`
|
||||
4. `data-raw/create_sbdv_plats.R` — builds `data/plats/plats.shp`
|
||||
5. `data-raw/create_venice_bndry.R` — builds `data/venice.rds`
|
||||
|
||||
## App tabs
|
||||
|
||||
| Tab | Content |
|
||||
|-----|---------|
|
||||
| About | Community description + photo |
|
||||
| Venice | City boundary map + facts table |
|
||||
| Beach | EPA beach locations map + helpful links |
|
||||
| Owners | Searchable owner map + table (filter by name, address, subdivision) |
|
||||
| Resources | Links to city services + PDF documents |
|
||||
|
||||
PDF documents are served from `www/docs/`.
|
||||
|
||||
## Running locally
|
||||
|
||||
Run inside the `rstudio` Docker container (see global CLAUDE.md):
|
||||
|
||||
```r
|
||||
shiny::runApp("/home/rstudio/projects/r/stAndrews")
|
||||
```
|
||||
|
||||
Or from RStudio with the project open: click **Run App**.
|
||||
|
||||
## Deployment
|
||||
|
||||
Deployed as a Shiny app. Path routing via the analytics gateway:
|
||||
- Add a route to `~/docker/gateway/Caddyfile` on the analytics VM if not already present.
|
||||
|
||||
## Old app
|
||||
|
||||
Previously deployed to shinyapps.io at `https://rob-wiederstein.shinyapps.io/stAndrews/`
|
||||
(account: `rob-wiederstein`, appId: 14173710). No longer maintained — app should be
|
||||
archived/deleted from the shinyapps.io dashboard. The local `rsconnect/` directory
|
||||
has been removed.
|
||||
|
||||
### Problems
|
||||
|
||||
**1. Geocoding was unreliable and required manual correction**
|
||||
|
||||
St. Andrews has three property types: standalone homes, villas (2- and 4-unit structures
|
||||
where each side is a separate unit), and 8-unit buildings. The 8-unit buildings have unit
|
||||
numbers (1–8) in the raw property records. Those unit numbers were stripped before
|
||||
geocoding, so all 8 units in a building returned a single shared coordinate. Google's
|
||||
geocoding was also only approximate — coordinates landed near addresses but not on top
|
||||
of the actual structures. The result was that every point had to be manually reviewed
|
||||
and dragged to its correct location in QGIS. This was labor-intensive and not repeatable.
|
||||
|
||||
**2. No update path**
|
||||
|
||||
The data pipeline was designed as a one-shot process with no way to refresh. Property
|
||||
ownership changes hands regularly — the data should be updated roughly weekly. The
|
||||
manual QGIS correction step makes this especially painful: any new or changed record
|
||||
would require re-running geocoding and repeating the manual editing. There is no
|
||||
mechanism to diff new records against existing ones or to carry forward previously
|
||||
corrected coordinates.
|
||||
|
||||
### Geographic data flow
|
||||
|
||||
There are three independent geographic data flows, all ending in WGS84 (EPSG:4326):
|
||||
|
||||
**1. Owner points** (`data-raw/main.R`)
|
||||
|
||||
Raw property records from the Sarasota County Property Appraiser (Excel) are filtered
|
||||
to the 13 St. Andrews subdivisions. Street addresses are parsed and assembled into
|
||||
geocodable strings, then sent to the Google geocoding API via `tidygeocoder`. Results
|
||||
are cached as `geotagged_street_addresses.rds` so the API isn't called twice. The
|
||||
geocoded points are written to `owners_raw.gpkg`. Because many units share a building
|
||||
address, they all land on the same coordinate — so the file is loaded into QGIS and
|
||||
points are manually dragged to their actual locations. The adjusted file
|
||||
(`owners_moved.gpkg`) is read back into R and saved as `data/owners.rds`.
|
||||
|
||||
**2. Subdivision boundary polygons** (`data-raw/create_sbdv_plats.R`)
|
||||
|
||||
A Sarasota County GIS shapefile (`data-raw/PlatBoundary/`) containing all county plat
|
||||
boundaries is read in, reprojected to WGS84, and filtered to the same 13 subdivision
|
||||
IDs. Subdivision names are cleaned up and the result is written to `data/plats/plats.shp`.
|
||||
|
||||
**3. Venice city boundary** (`data-raw/create_venice_bndry.R`)
|
||||
|
||||
A Sarasota County boundary shapefile (`data-raw/SarasotaCountyBoundary/`) is filtered
|
||||
to the City of Venice (`municipali == "CV"`), keeping only the main polygon (acreage >
|
||||
2500 to drop small outliers). It is simplified with `rmapshaper`, interior holes are
|
||||
removed, reprojected to WGS84, and saved as `data/venice.rds`.
|
||||
|
||||
## New app — decisions
|
||||
|
||||
**Same folder/repo.** The UI in `app.R` is already good — tabs, maps, and layout don't
|
||||
need to change. What needs replacing is entirely in `data-raw/`: the pipeline that
|
||||
produces `data/owners.rds`. The app just consumes that file. Rebuild the pipeline,
|
||||
keep the app.
|
||||
|
||||
**Feasibility hinges on the join.** The geometry side is straightforward — filter
|
||||
building footprints to the subdivision boundary, compute centroids, done. The hard part
|
||||
is linking those geometries to SCPA property records. Two approaches worth investigating:
|
||||
|
||||
- **Address join** — if the footprint layer carries situs addresses in the same format
|
||||
as the SCPA records, a direct join works. If formatting differs, fuzzy matching may
|
||||
be needed.
|
||||
- **Account number join** — the SCPA assigns an account number to each property. It's
|
||||
worth checking whether account numbers follow the *property* (stable, ideal join key)
|
||||
or the *owner* (changes on sale, less useful). If account numbers are property-stable
|
||||
and also appear in the footprint layer, this would be the cleanest and most reliable
|
||||
join key — especially for multi-unit buildings where address disambiguation is messy.
|
||||
|
||||
Also needs confirming: whether the footprint polygons exist at the **unit level** or
|
||||
only at the **building level**. If only building-level, the stacking problem from the
|
||||
old pipeline reappears.
|
||||
|
||||
Data inspection will resolve all of this.
|
||||
|
||||
## New app — proposal
|
||||
|
||||
Instead of geocoding addresses, use the Sarasota County GIS building footprint layer,
|
||||
which contains polygon outlines of every structure. Key advantages:
|
||||
|
||||
- **No geocoding API needed.** Building outlines are already precisely positioned over
|
||||
actual structures. Deriving centroids from the polygons gives accurate, stable
|
||||
coordinates for every unit — far better than approximate Google geocoding results.
|
||||
- **Handles multi-unit buildings cleanly.** Each unit in an 8-unit building has its own
|
||||
footprint polygon, so each gets its own distinct centroid. No stripping of unit numbers,
|
||||
no stacking of points, no manual QGIS correction.
|
||||
- **No new construction to worry about.** St. Andrews is a built-out community with no
|
||||
active development, so the building footprint layer is effectively static.
|
||||
|
||||
**Proposed pipeline:**
|
||||
|
||||
1. Download the Sarasota County building footprint GIS layer.
|
||||
2. Spatially filter it to the St. Andrews subdivision boundary (already have
|
||||
`data/plats/plats.shp`).
|
||||
3. Compute centroids for each building footprint polygon.
|
||||
4. Join centroids to property records from the SCPA using address as the key.
|
||||
5. Save the joined dataset as the owner points layer.
|
||||
|
||||
**Update path:**
|
||||
|
||||
Because the geometry is fixed (building centroids don't change), only the ownership
|
||||
attributes need refreshing. The SCPA property records can be re-downloaded weekly and
|
||||
re-joined to the stable centroid table. This makes weekly updates a simple scripted
|
||||
operation with no manual steps.
|
||||
|
||||
## Notes
|
||||
|
||||
- Geocoding used Google API via `tidygeocoder`; results cached in `data-raw/geotagged_street_addresses.rds` to avoid re-calling the API.
|
||||
- Point deduplication (multiple units at same address) was done manually in QGIS — not scripted. `owners_moved.gpkg` is the authoritative geocoded dataset.
|
||||
- `data-raw/` is gitignored except for the shapefiles in `data-raw/PlatBoundary/` and `data-raw/SarasotaCountyBoundary/` which are committed.
|
||||
Reference in New Issue
Block a user