oneie
Version:
Build apps, websites, and AI agents in English. Zero-interaction setup for AI agents (Claude Code, Cursor, Windsurf). Download to your computer, run in the cloud, deploy to the edge. Open source and free forever.
1,829 lines (1,565 loc) • 58.9 kB
Markdown
---
title: Astro Effect Complete Vision
dimension: knowledge
category: astro-effect-complete-vision.md
tags: architecture
related_dimensions: events, things
scope: global
created: 2025-11-03
updated: 2025-11-03
version: 1.0.0
ai_context: |
This document is part of the knowledge dimension in the astro-effect-complete-vision.md category.
Location: one/knowledge/astro-effect-complete-vision.md
Purpose: Documents complete vision: astro + effect.ts in 5 layers
Related dimensions: events, things
For AI agents: Read this to understand astro effect complete vision.
---
# Complete Vision: Astro + Effect.ts in 5 Layers
## Overview: Progressive Enhancement Architecture
This document shows **real-world examples** of building with Astro + Effect.ts, progressing through 5 layers of complexity. Each layer builds on the previous, showing when and why to add each piece.
**Foundation:** Read `astro-effect-simple-architecture.md` first for the core philosophy.
## The 5 Layers
```
Layer 1: Static Content (Astro + Content Collections)
↓
Layer 2: Validation (+ Zod schemas + Effect.ts)
↓
Layer 3: Client State (+ React islands)
↓
Layer 4: Data Persistence (+ REST API)
↓
Layer 5: Full-Stack App (+ Database + Auth)
```
**Golden Rule:** Start at Layer 1. Only move to the next layer when you need it.
---
## Layer 1: Static Content (Blog Example)
**Use Case:** Marketing site, blog, documentation - anything that's mostly read-only.
**What You Need:**
- Astro pages (routing)
- Content collections (markdown/YAML)
- shadcn components (UI)
**What You DON'T Need:**
- ❌ State management
- ❌ Database
- ❌ API
- ❌ Authentication
### Complete File Structure
```
src/
├── pages/
│ ├── index.astro # Homepage
│ ├── blog/
│ │ ├── index.astro # Blog list
│ │ └── [...slug].astro # Blog post detail
│ └── about.astro # About page
│
├── content/
│ ├── config.ts # Content schemas
│ └── blog/
│ ├── first-post.md
│ ├── second-post.md
│ └── third-post.md
│
├── components/
│ ├── BlogCard.tsx # Post preview card
│ ├── BlogPost.tsx # Full post display
│ └── ui/ # shadcn components
│ ├── card.tsx
│ ├── button.tsx
│ └── badge.tsx
│
└── layouts/
└── Layout.astro # Base layout
```
### Schema Definition
```typescript
// src/content/config.ts
import { defineCollection, z } from "astro:content";
const blogCollection = defineCollection({
type: "content",
schema: z.object({
title: z.string(),
description: z.string(),
author: z.string(),
date: z.date(),
tags: z.array(z.string()).default([]),
featured: z.boolean().default(false),
}),
});
export const collections = { blog: blogCollection };
```
### Content Files
```markdown
---
# src/content/blog/first-post.md
title: "Getting Started with Astro"
description: "Learn how to build lightning-fast websites"
author: "Alice Developer"
date: 2025-01-15
tags: ["astro", "web-dev"]
featured: true
---
# Getting Started with Astro
Astro is a modern static site generator that lets you build faster websites...
## Key Features
- **Islands Architecture**: Ship less JavaScript
- **Content Collections**: Type-safe content
- **Framework Agnostic**: Use React, Vue, Svelte, or none
```
### List Page
```astro
---
// src/pages/blog/index.astro
import Layout from "@/layouts/Layout.astro";
import { getCollection } from "astro:content";
import BlogCard from "@/components/BlogCard.tsx";
// Get all blog posts, sorted by date
const posts = (await getCollection("blog"))
.sort((a, b) => b.data.date.valueOf() - a.data.date.valueOf());
const featuredPosts = posts.filter(p => p.data.featured);
const regularPosts = posts.filter(p => !p.data.featured);
---
<Layout title="Blog">
<div class="container py-12">
<h1 class="text-5xl font-bold mb-4">Blog</h1>
<p class="text-xl text-gray-600 mb-12">
Thoughts on web development, design, and technology
</p>
{featuredPosts.length > 0 && (
<section class="mb-12">
<h2 class="text-3xl font-bold mb-6">Featured Posts</h2>
<div class="grid grid-cols-1 md:grid-cols-2 gap-6">
{featuredPosts.map(post => (
<BlogCard post={post} featured={true} />
))}
</div>
</section>
)}
<section>
<h2 class="text-3xl font-bold mb-6">All Posts</h2>
<div class="grid grid-cols-1 md:grid-cols-3 gap-6">
{regularPosts.map(post => (
<BlogCard post={post} />
))}
</div>
</section>
</div>
</Layout>
```
### Detail Page
```astro
---
// src/pages/blog/[...slug].astro
import Layout from "@/layouts/Layout.astro";
import { getCollection, type CollectionEntry } from "astro:content";
import { Badge } from "@/components/ui/badge";
export async function getStaticPaths() {
const posts = await getCollection("blog");
return posts.map(post => ({
params: { slug: post.slug },
props: { post },
}));
}
type Props = { post: CollectionEntry<"blog"> };
const { post } = Astro.props;
const { Content } = await post.render();
---
<Layout title={post.data.title}>
<article class="container max-w-4xl py-12">
<header class="mb-8">
<h1 class="text-5xl font-bold mb-4">{post.data.title}</h1>
<p class="text-xl text-gray-600 mb-4">{post.data.description}</p>
<div class="flex items-center gap-4 text-sm text-gray-500 mb-4">
<span>By {post.data.author}</span>
<span>•</span>
<time datetime={post.data.date.toISOString()}>
{post.data.date.toLocaleDateString()}
</time>
</div>
<div class="flex gap-2">
{post.data.tags.map(tag => (
<Badge variant="secondary">{tag}</Badge>
))}
</div>
</header>
<div class="prose prose-lg max-w-none">
<Content />
</div>
</article>
</Layout>
```
### Component
```tsx
// src/components/BlogCard.tsx
import { Card, CardHeader, CardTitle, CardDescription, CardContent } from "@/components/ui/card";
import { Badge } from "@/components/ui/badge";
import type { CollectionEntry } from "astro:content";
type Props = {
post: CollectionEntry<"blog">;
featured?: boolean;
};
export function BlogCard({ post, featured = false }: Props) {
return (
<a href={`/blog/${post.slug}`} class="block transition hover:scale-105">
<Card className={featured ? "border-2 border-blue-500" : ""}>
<CardHeader>
<CardTitle>{post.data.title}</CardTitle>
<CardDescription>{post.data.description}</CardDescription>
</CardHeader>
<CardContent>
<div class="flex items-center justify-between text-sm text-gray-500">
<span>{post.data.author}</span>
<time datetime={post.data.date.toISOString()}>
{post.data.date.toLocaleDateString()}
</time>
</div>
<div class="flex gap-2 mt-3">
{post.data.tags.slice(0, 3).map(tag => (
<Badge key={tag} variant="outline">{tag}</Badge>
))}
</div>
</CardContent>
</Card>
</a>
);
}
```
**Layer 1 Result:** Static blog with type-safe content, beautiful UI, and zero JavaScript overhead.
---
## Layer 2: Validation (Product Catalog Example)
**Use Case:** E-commerce product catalog, pricing calculator - anything needing input validation.
**Added to Layer 1:**
- ✅ Effect.ts services (validation logic)
- ✅ Zod refinements (advanced validation)
**Still Don't Need:**
- ❌ Client state
- ❌ Database
- ❌ API
### Complete File Structure
```
src/
├── pages/
│ ├── products/
│ │ ├── index.astro # Product list
│ │ └── [slug].astro # Product detail
│ └── calculator.astro # Price calculator
│
├── content/
│ ├── config.ts
│ └── products/
│ ├── product-1.yaml
│ └── product-2.yaml
│
├── components/
│ ├── ProductCard.tsx
│ ├── PriceCalculator.tsx # Uses Effect.ts validation
│ └── ui/
│
└── lib/
└── services/
└── productService.ts # Effect.ts validation logic
```
### Enhanced Schema with Validation
```typescript
// src/content/config.ts
import { defineCollection, z } from "astro:content";
const productCollection = defineCollection({
type: "data",
schema: z.object({
name: z.string().min(1, "Product name required"),
slug: z.string().regex(/^[a-z0-9-]+$/, "Invalid slug format"),
description: z.string(),
price: z.number().positive("Price must be positive"),
currency: z.enum(["USD", "EUR", "GBP"]).default("USD"),
category: z.enum(["software", "hardware", "service"]),
inStock: z.boolean().default(true),
features: z.array(z.string()).min(1, "At least one feature required"),
specifications: z.record(z.string(), z.any()).optional(),
}).refine(
data => data.price > 0 || !data.inStock,
{ message: "In-stock products must have a positive price" }
),
});
export const collections = { products: productCollection };
```
### Product Content
```yaml
# src/content/products/pro-plan.yaml
name: "Pro Plan"
slug: "pro-plan"
description: "Professional features for growing teams"
price: 49.99
currency: "USD"
category: "software"
inStock: true
features:
- "Unlimited projects"
- "Advanced analytics"
- "Priority support"
- "Custom integrations"
specifications:
users: "Up to 50"
storage: "1TB"
apiCalls: "100,000/month"
```
### Effect.ts Service
```typescript
// src/lib/services/productService.ts
import { Effect, Data } from "effect";
// Tagged errors
export class ValidationError extends Data.TaggedError("ValidationError")<{
field: string;
message: string;
}> {}
export class PriceError extends Data.TaggedError("PriceError")<{
reason: string;
}> {}
// Validation functions
export const validateQuantity = (
quantity: number
): Effect.Effect<number, ValidationError> =>
Effect.gen(function* () {
if (!Number.isInteger(quantity)) {
return yield* new ValidationError({
field: "quantity",
message: "Quantity must be an integer",
});
}
if (quantity < 1) {
return yield* new ValidationError({
field: "quantity",
message: "Quantity must be at least 1",
});
}
if (quantity > 1000) {
return yield* new ValidationError({
field: "quantity",
message: "Quantity cannot exceed 1000",
});
}
return quantity;
});
export const calculatePrice = (
basePrice: number,
quantity: number,
discountCode?: string
): Effect.Effect<{ total: number; discount: number }, PriceError | ValidationError> =>
Effect.gen(function* () {
// Validate quantity first
const validQuantity = yield* validateQuantity(quantity);
if (basePrice <= 0) {
return yield* new PriceError({ reason: "Invalid base price" });
}
let discount = 0;
// Apply volume discount
if (validQuantity >= 10) discount += 0.1; // 10% off
if (validQuantity >= 50) discount += 0.05; // Additional 5% off
// Apply promo code
if (discountCode === "SAVE20") discount += 0.2;
if (discountCode === "SAVE10") discount += 0.1;
const subtotal = basePrice * validQuantity;
const discountAmount = subtotal * discount;
const total = subtotal - discountAmount;
return { total, discount: discountAmount };
});
// Composable validation pipeline
export const validateAndPrice = (
product: { price: number; inStock: boolean },
quantity: number,
discountCode?: string
): Effect.Effect<{ total: number; discount: number }, ValidationError | PriceError> =>
Effect.gen(function* () {
if (!product.inStock) {
return yield* new PriceError({ reason: "Product out of stock" });
}
return yield* calculatePrice(product.price, quantity, discountCode);
});
```
### Calculator Component (Client Island)
```tsx
// src/components/PriceCalculator.tsx
import { useState } from "react";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { Label } from "@/components/ui/label";
import { Card, CardHeader, CardTitle, CardContent } from "@/components/ui/card";
import { Alert, AlertDescription } from "@/components/ui/alert";
import { Effect } from "effect";
import { calculatePrice, type ValidationError, type PriceError } from "@/lib/services/productService";
type Props = {
basePrice: number;
};
export function PriceCalculator({ basePrice }: Props) {
const [quantity, setQuantity] = useState(1);
const [discountCode, setDiscountCode] = useState("");
const [result, setResult] = useState<{ total: number; discount: number } | null>(null);
const [error, setError] = useState<string | null>(null);
const handleCalculate = async () => {
setError(null);
const effect = calculatePrice(basePrice, quantity, discountCode || undefined);
const result = await Effect.runPromise(
Effect.either(effect)
);
if (result._tag === "Left") {
const err = result.left;
if (err._tag === "ValidationError") {
setError(`${err.field}: ${err.message}`);
} else if (err._tag === "PriceError") {
setError(err.reason);
}
setResult(null);
} else {
setResult(result.right);
}
};
return (
<Card>
<CardHeader>
<CardTitle>Price Calculator</CardTitle>
</CardHeader>
<CardContent className="space-y-4">
<div>
<Label htmlFor="quantity">Quantity</Label>
<Input
id="quantity"
type="number"
min="1"
value={quantity}
onChange={(e) => setQuantity(Number(e.target.value))}
/>
</div>
<div>
<Label htmlFor="discount">Discount Code (optional)</Label>
<Input
id="discount"
type="text"
placeholder="SAVE20"
value={discountCode}
onChange={(e) => setDiscountCode(e.target.value.toUpperCase())}
/>
</div>
<Button onClick={handleCalculate} className="w-full">
Calculate Price
</Button>
{error && (
<Alert variant="destructive">
<AlertDescription>{error}</AlertDescription>
</Alert>
)}
{result && (
<div className="p-4 bg-green-50 rounded-lg">
<p className="text-sm text-gray-600">Discount: ${result.discount.toFixed(2)}</p>
<p className="text-2xl font-bold text-green-600">
Total: ${result.total.toFixed(2)}
</p>
</div>
)}
</CardContent>
</Card>
);
}
```
### Product Page with Calculator
```astro
---
// src/pages/products/[slug].astro
import Layout from "@/layouts/Layout.astro";
import { getCollection } from "astro:content";
import PriceCalculator from "@/components/PriceCalculator.tsx";
import { Badge } from "@/components/ui/badge";
export async function getStaticPaths() {
const products = await getCollection("products");
return products.map(product => ({
params: { slug: product.data.slug },
props: { product },
}));
}
const { product } = Astro.props;
---
<Layout title={product.data.name}>
<div class="container max-w-6xl py-12">
<div class="grid grid-cols-1 md:grid-cols-2 gap-12">
<!-- Product Info -->
<div>
<h1 class="text-4xl font-bold mb-4">{product.data.name}</h1>
<p class="text-xl text-gray-600 mb-6">{product.data.description}</p>
<div class="mb-6">
<Badge variant={product.data.inStock ? "default" : "destructive"}>
{product.data.inStock ? "In Stock" : "Out of Stock"}
</Badge>
<Badge variant="outline" class="ml-2">{product.data.category}</Badge>
</div>
<div class="mb-6">
<h2 class="text-2xl font-bold mb-3">Features</h2>
<ul class="list-disc list-inside space-y-2">
{product.data.features.map(feature => (
<li>{feature}</li>
))}
</ul>
</div>
{product.data.specifications && (
<div>
<h2 class="text-2xl font-bold mb-3">Specifications</h2>
<dl class="space-y-2">
{Object.entries(product.data.specifications).map(([key, value]) => (
<div class="flex justify-between">
<dt class="font-medium">{key}:</dt>
<dd class="text-gray-600">{value}</dd>
</div>
))}
</dl>
</div>
)}
</div>
<!-- Price Calculator -->
<div>
<PriceCalculator basePrice={product.data.price} client:load />
</div>
</div>
</div>
</Layout>
```
**Layer 2 Result:** Product catalog with sophisticated validation, pricing logic, and error handling - all type-safe.
---
## Layer 3: Client State (Shopping Cart Example)
**Use Case:** Shopping cart, interactive forms, multi-step wizards - anything needing client-side state.
**Added to Layer 2:**
- ✅ React state hooks (useState, useReducer)
- ✅ Client-side interactivity
- ✅ nanostores (for global client state)
**Still Don't Need:**
- ❌ Database
- ❌ API (state is ephemeral)
### Complete File Structure
```
src/
├── pages/
│ ├── shop/
│ │ ├── index.astro # Product listing
│ │ ├── [slug].astro # Product detail
│ │ └── cart.astro # Shopping cart page
│ └── checkout.astro # Checkout flow
│
├── content/
│ └── products/
│
├── components/
│ ├── ProductCard.tsx
│ ├── AddToCartButton.tsx # Client island
│ ├── ShoppingCart.tsx # Client island
│ └── CartSummary.tsx # Client island
│
├── lib/
│ ├── services/
│ │ ├── productService.ts
│ │ └── cartService.ts # Effect.ts cart logic
│ └── stores/
│ └── cart.ts # nanostores global state
```
### Cart Store (nanostores)
```typescript
// src/lib/stores/cart.ts
import { atom, map } from "nanostores";
export type CartItem = {
productId: string;
name: string;
price: number;
quantity: number;
};
export const cartItems = map<Record<string, CartItem>>({});
export const cartCount = atom(0);
// Actions
export function addToCart(item: Omit<CartItem, "quantity">) {
const current = cartItems.get();
const existing = current[item.productId];
if (existing) {
cartItems.setKey(item.productId, {
...existing,
quantity: existing.quantity + 1,
});
} else {
cartItems.setKey(item.productId, { ...item, quantity: 1 });
}
updateCartCount();
}
export function removeFromCart(productId: string) {
const current = cartItems.get();
const { [productId]: removed, ...rest } = current;
cartItems.set(rest);
updateCartCount();
}
export function updateQuantity(productId: string, quantity: number) {
if (quantity <= 0) {
removeFromCart(productId);
} else {
const current = cartItems.get();
const item = current[productId];
if (item) {
cartItems.setKey(productId, { ...item, quantity });
}
}
updateCartCount();
}
export function clearCart() {
cartItems.set({});
cartCount.set(0);
}
function updateCartCount() {
const items = Object.values(cartItems.get());
const total = items.reduce((sum, item) => sum + item.quantity, 0);
cartCount.set(total);
}
```
### Cart Service (Effect.ts)
```typescript
// src/lib/services/cartService.ts
import { Effect, Data } from "effect";
import type { CartItem } from "@/lib/stores/cart";
export class CartError extends Data.TaggedError("CartError")<{
reason: string;
}> {}
export const validateCartItem = (
item: CartItem
): Effect.Effect<CartItem, CartError> =>
Effect.gen(function* () {
if (item.quantity < 1) {
return yield* new CartError({ reason: "Quantity must be at least 1" });
}
if (item.quantity > 99) {
return yield* new CartError({ reason: "Quantity cannot exceed 99" });
}
if (item.price <= 0) {
return yield* new CartError({ reason: "Invalid price" });
}
return item;
});
export const calculateCartTotal = (
items: CartItem[]
): Effect.Effect<{ subtotal: number; tax: number; total: number }, CartError> =>
Effect.gen(function* () {
// Validate all items
const validatedItems = yield* Effect.all(
items.map(validateCartItem),
{ concurrency: "unbounded" }
);
const subtotal = validatedItems.reduce(
(sum, item) => sum + item.price * item.quantity,
0
);
const tax = subtotal * 0.08; // 8% tax
const total = subtotal + tax;
return { subtotal, tax, total };
});
export const applyDiscount = (
total: number,
code: string
): Effect.Effect<{ discount: number; finalTotal: number }, CartError> =>
Effect.gen(function* () {
let discount = 0;
switch (code.toUpperCase()) {
case "SAVE10":
discount = total * 0.1;
break;
case "SAVE20":
discount = total * 0.2;
break;
case "FREESHIP":
discount = 10; // Free shipping
break;
default:
return yield* new CartError({ reason: "Invalid discount code" });
}
return {
discount,
finalTotal: total - discount,
};
});
```
### Add to Cart Button
```tsx
// src/components/AddToCartButton.tsx
import { useState } from "react";
import { Button } from "@/components/ui/button";
import { addToCart } from "@/lib/stores/cart";
import { ShoppingCart, Check } from "lucide-react";
type Props = {
productId: string;
name: string;
price: number;
};
export function AddToCartButton({ productId, name, price }: Props) {
const [added, setAdded] = useState(false);
const handleAdd = () => {
addToCart({ productId, name, price });
setAdded(true);
setTimeout(() => setAdded(false), 2000);
};
return (
<Button onClick={handleAdd} className="w-full" disabled={added}>
{added ? (
<>
<Check className="mr-2 h-4 w-4" />
Added!
</>
) : (
<>
<ShoppingCart className="mr-2 h-4 w-4" />
Add to Cart
</>
)}
</Button>
);
}
```
### Shopping Cart Component
```tsx
// src/components/ShoppingCart.tsx
import { useStore } from "@nanostores/react";
import { cartItems, removeFromCart, updateQuantity } from "@/lib/stores/cart";
import { calculateCartTotal } from "@/lib/services/cartService";
import { Card, CardHeader, CardTitle, CardContent } from "@/components/ui/card";
import { Button } from "@/components/ui/button";
import { Input } from "@/components/ui/input";
import { Trash2 } from "lucide-react";
import { useEffect, useState } from "react";
import { Effect } from "effect";
export function ShoppingCart() {
const $cartItems = useStore(cartItems);
const items = Object.values($cartItems);
const [totals, setTotals] = useState<{
subtotal: number;
tax: number;
total: number;
} | null>(null);
useEffect(() => {
const calculateTotals = async () => {
if (items.length === 0) {
setTotals(null);
return;
}
const result = await Effect.runPromise(
Effect.either(calculateCartTotal(items))
);
if (result._tag === "Right") {
setTotals(result.right);
}
};
calculateTotals();
}, [$cartItems]);
if (items.length === 0) {
return (
<Card>
<CardContent className="p-12 text-center">
<p className="text-gray-500">Your cart is empty</p>
</CardContent>
</Card>
);
}
return (
<Card>
<CardHeader>
<CardTitle>Shopping Cart ({items.length} items)</CardTitle>
</CardHeader>
<CardContent className="space-y-4">
{items.map(item => (
<div key={item.productId} className="flex items-center gap-4 p-4 border rounded">
<div className="flex-1">
<h3 className="font-semibold">{item.name}</h3>
<p className="text-sm text-gray-600">${item.price.toFixed(2)} each</p>
</div>
<div className="flex items-center gap-2">
<Input
type="number"
min="1"
max="99"
value={item.quantity}
onChange={(e) => updateQuantity(item.productId, Number(e.target.value))}
className="w-20"
/>
</div>
<div className="font-bold">
${(item.price * item.quantity).toFixed(2)}
</div>
<Button
variant="ghost"
size="icon"
onClick={() => removeFromCart(item.productId)}
>
<Trash2 className="h-4 w-4" />
</Button>
</div>
))}
{totals && (
<div className="border-t pt-4 space-y-2">
<div className="flex justify-between text-sm">
<span>Subtotal:</span>
<span>${totals.subtotal.toFixed(2)}</span>
</div>
<div className="flex justify-between text-sm">
<span>Tax (8%):</span>
<span>${totals.tax.toFixed(2)}</span>
</div>
<div className="flex justify-between text-lg font-bold">
<span>Total:</span>
<span>${totals.total.toFixed(2)}</span>
</div>
</div>
)}
<Button className="w-full" size="lg">
Proceed to Checkout
</Button>
</CardContent>
</Card>
);
}
```
### Cart Page
```astro
---
// src/pages/shop/cart.astro
import Layout from "@/layouts/Layout.astro";
import ShoppingCart from "@/components/ShoppingCart.tsx";
---
<Layout title="Shopping Cart">
<div class="container max-w-4xl py-12">
<h1 class="text-4xl font-bold mb-8">Shopping Cart</h1>
<ShoppingCart client:load />
</div>
</Layout>
```
### Product Page with Cart Button
```astro
---
// src/pages/shop/[slug].astro
import Layout from "@/layouts/Layout.astro";
import { getCollection } from "astro:content";
import AddToCartButton from "@/components/AddToCartButton.tsx";
export async function getStaticPaths() {
const products = await getCollection("products");
return products.map(product => ({
params: { slug: product.data.slug },
props: { product },
}));
}
const { product } = Astro.props;
---
<Layout title={product.data.name}>
<div class="container max-w-4xl py-12">
<h1 class="text-4xl font-bold mb-4">{product.data.name}</h1>
<p class="text-xl text-gray-600 mb-6">{product.data.description}</p>
<div class="text-3xl font-bold text-green-600 mb-6">
${product.data.price.toFixed(2)}
</div>
<div class="max-w-sm">
<AddToCartButton
productId={product.id}
name={product.data.name}
price={product.data.price}
client:load
/>
</div>
</div>
</Layout>
```
**Layer 3 Result:** Interactive shopping cart with client-side state management, all validated with Effect.ts.
---
## Layer 4: Data Persistence (Multi-Source Migration Example)
**Use Case:** Data migrations, external API integration, CMS syncing - anything that fetches/transforms data from external sources.
**Added to Layer 3:**
- ✅ REST API integration
- ✅ External data fetching
- ✅ Data transformation pipelines
**Still Don't Need:**
- ❌ Your own database (reading from external sources)
### Complete File Structure
```
src/
├── pages/
│ ├── migrate/
│ │ ├── index.astro # Migration dashboard
│ │ └── [source].astro # Source-specific migration
│ └── api/
│ └── migrate.ts # API endpoint for migrations
│
├── lib/
│ └── services/
│ ├── migrationService.ts # Effect.ts migration logic
│ ├── githubAdapter.ts # GitHub API adapter
│ ├── notionAdapter.ts # Notion API adapter
│ └── stripeAdapter.ts # Stripe API adapter
│
└── components/
└── MigrationStatus.tsx # Real-time status display
```
### Migration Service (Effect.ts)
```typescript
// src/lib/services/migrationService.ts
import { Effect, Data, Context } from "effect";
// Tagged errors
export class FetchError extends Data.TaggedError("FetchError")<{
source: string;
reason: string;
}> {}
export class TransformError extends Data.TaggedError("TransformError")<{
item: string;
reason: string;
}> {}
export class ValidationError extends Data.TaggedError("ValidationError")<{
field: string;
message: string;
}> {}
// Data source interface
export interface DataSource {
readonly name: string;
readonly fetch: () => Effect.Effect<unknown[], FetchError>;
readonly transform: (data: unknown) => Effect.Effect<MigratedItem, TransformError>;
readonly validate: (item: MigratedItem) => Effect.Effect<MigratedItem, ValidationError>;
}
export const DataSource = Context.GenericTag<DataSource>("DataSource");
// Migrated item schema
export type MigratedItem = {
id: string;
title: string;
content: string;
author: string;
createdAt: Date;
tags: string[];
metadata: Record<string, any>;
};
// Migration pipeline
export const migrateFromSource = (
source: DataSource
): Effect.Effect<MigratedItem[], FetchError | TransformError | ValidationError> =>
Effect.gen(function* () {
console.log(`Starting migration from ${source.name}...`);
// Step 1: Fetch raw data
const rawData = yield* source.fetch();
console.log(`Fetched ${rawData.length} items from ${source.name}`);
// Step 2: Transform each item
const transformed = yield* Effect.all(
rawData.map(item => source.transform(item)),
{ concurrency: 5 } // Process 5 items at a time
);
console.log(`Transformed ${transformed.length} items`);
// Step 3: Validate each item
const validated = yield* Effect.all(
transformed.map(item => source.validate(item)),
{ concurrency: "unbounded" }
);
console.log(`Validated ${validated.length} items`);
return validated;
});
// Batch migration from multiple sources
export const migrateFromMultipleSources = (
sources: DataSource[]
): Effect.Effect<
Map<string, MigratedItem[]>,
FetchError | TransformError | ValidationError
> =>
Effect.gen(function* () {
const results = yield* Effect.all(
sources.map(source =>
migrateFromSource(source).pipe(
Effect.map(items => [source.name, items] as const)
)
),
{ concurrency: 3 } // Process 3 sources at a time
);
return new Map(results);
});
// Validation helper
export const validateMigratedItem = (
item: MigratedItem
): Effect.Effect<MigratedItem, ValidationError> =>
Effect.gen(function* () {
if (!item.id?.trim()) {
return yield* new ValidationError({
field: "id",
message: "ID is required",
});
}
if (!item.title?.trim()) {
return yield* new ValidationError({
field: "title",
message: "Title is required",
});
}
if (item.tags.length === 0) {
return yield* new ValidationError({
field: "tags",
message: "At least one tag required",
});
}
return item;
});
```
### GitHub Adapter
```typescript
// src/lib/services/githubAdapter.ts
import { Effect } from "effect";
import { FetchError, TransformError, validateMigratedItem, type MigratedItem, type DataSource } from "./migrationService";
type GitHubIssue = {
id: number;
number: number;
title: string;
body: string;
user: { login: string };
created_at: string;
labels: Array<{ name: string }>;
};
export const GitHubDataSource: DataSource = {
name: "GitHub",
fetch: (): Effect.Effect<GitHubIssue[], FetchError> =>
Effect.tryPromise({
try: async () => {
const response = await fetch(
"https://api.github.com/repos/owner/repo/issues",
{
headers: {
Authorization: `token ${import.meta.env.GITHUB_TOKEN}`,
Accept: "application/vnd.github.v3+json",
},
}
);
if (!response.ok) {
throw new Error(`GitHub API error: ${response.statusText}`);
}
return response.json();
},
catch: (error) =>
new FetchError({
source: "GitHub",
reason: error instanceof Error ? error.message : "Unknown error",
}),
}),
transform: (data: unknown): Effect.Effect<MigratedItem, TransformError> =>
Effect.try({
try: () => {
const issue = data as GitHubIssue;
return {
id: `github-${issue.number}`,
title: issue.title,
content: issue.body || "",
author: issue.user.login,
createdAt: new Date(issue.created_at),
tags: issue.labels.map(l => l.name),
metadata: {
source: "github",
issueNumber: issue.number,
url: `https://github.com/owner/repo/issues/${issue.number}`,
},
};
},
catch: (error) =>
new TransformError({
item: JSON.stringify(data),
reason: error instanceof Error ? error.message : "Transform failed",
}),
}),
validate: validateMigratedItem,
};
```
### Notion Adapter
```typescript
// src/lib/services/notionAdapter.ts
import { Effect } from "effect";
import { FetchError, TransformError, validateMigratedItem, type MigratedItem, type DataSource } from "./migrationService";
type NotionPage = {
id: string;
properties: {
Title: { title: Array<{ plain_text: string }> };
Tags: { multi_select: Array<{ name: string }> };
Created: { created_time: string };
};
url: string;
};
export const NotionDataSource: DataSource = {
name: "Notion",
fetch: (): Effect.Effect<NotionPage[], FetchError> =>
Effect.tryPromise({
try: async () => {
const response = await fetch(
"https://api.notion.com/v1/databases/DATABASE_ID/query",
{
method: "POST",
headers: {
Authorization: `Bearer ${import.meta.env.NOTION_TOKEN}`,
"Notion-Version": "2022-06-28",
"Content-Type": "application/json",
},
}
);
if (!response.ok) {
throw new Error(`Notion API error: ${response.statusText}`);
}
const json = await response.json();
return json.results;
},
catch: (error) =>
new FetchError({
source: "Notion",
reason: error instanceof Error ? error.message : "Unknown error",
}),
}),
transform: (data: unknown): Effect.Effect<MigratedItem, TransformError> =>
Effect.try({
try: () => {
const page = data as NotionPage;
return {
id: `notion-${page.id}`,
title: page.properties.Title.title[0]?.plain_text || "Untitled",
content: "", // Would need to fetch page content separately
author: "notion-import",
createdAt: new Date(page.properties.Created.created_time),
tags: page.properties.Tags.multi_select.map(t => t.name),
metadata: {
source: "notion",
pageId: page.id,
url: page.url,
},
};
},
catch: (error) =>
new TransformError({
item: JSON.stringify(data),
reason: error instanceof Error ? error.message : "Transform failed",
}),
}),
validate: validateMigratedItem,
};
```
### Migration Page
```astro
---
// src/pages/migrate/index.astro
import Layout from "@/layouts/Layout.astro";
import { Button } from "@/components/ui/button";
import { Card, CardHeader, CardTitle, CardContent } from "@/components/ui/card";
import MigrationStatus from "@/components/MigrationStatus.tsx";
const sources = [
{ name: "GitHub", description: "Import issues as blog posts", icon: "📦" },
{ name: "Notion", description: "Import database pages", icon: "📝" },
{ name: "Stripe", description: "Import customer data", icon: "💳" },
];
---
<Layout title="Data Migration">
<div class="container max-w-6xl py-12">
<h1 class="text-4xl font-bold mb-4">Data Migration</h1>
<p class="text-xl text-gray-600 mb-12">
Import content from external sources
</p>
<div class="grid grid-cols-1 md:grid-cols-3 gap-6 mb-12">
{sources.map(source => (
<Card>
<CardHeader>
<CardTitle class="flex items-center gap-2">
<span class="text-3xl">{source.icon}</span>
{source.name}
</CardTitle>
</CardHeader>
<CardContent>
<p class="text-sm text-gray-600 mb-4">{source.description}</p>
<Button asChild class="w-full">
<a href={`/migrate/${source.name.toLowerCase()}`}>
Start Migration
</a>
</Button>
</CardContent>
</Card>
))}
</div>
<MigrationStatus client:load />
</div>
</Layout>
```
### Migration Status Component
```tsx
// src/components/MigrationStatus.tsx
import { useState } from "react";
import { Card, CardHeader, CardTitle, CardContent } from "@/components/ui/card";
import { Button } from "@/components/ui/button";
import { Alert, AlertDescription } from "@/components/ui/alert";
import { Effect } from "effect";
import { migrateFromMultipleSources } from "@/lib/services/migrationService";
import { GitHubDataSource } from "@/lib/services/githubAdapter";
import { NotionDataSource } from "@/lib/services/notionAdapter";
type MigrationState = "idle" | "running" | "success" | "error";
export function MigrationStatus() {
const [state, setState] = useState<MigrationState>("idle");
const [results, setResults] = useState<Map<string, any[]> | null>(null);
const [error, setError] = useState<string | null>(null);
const runMigration = async () => {
setState("running");
setError(null);
const sources = [GitHubDataSource, NotionDataSource];
const result = await Effect.runPromise(
Effect.either(migrateFromMultipleSources(sources))
);
if (result._tag === "Left") {
const err = result.left;
setError(`Migration failed: ${err._tag} - ${JSON.stringify(err)}`);
setState("error");
} else {
setResults(result.right);
setState("success");
}
};
return (
<Card>
<CardHeader>
<CardTitle>Migration Status</CardTitle>
</CardHeader>
<CardContent className="space-y-4">
<Button
onClick={runMigration}
disabled={state === "running"}
className="w-full"
>
{state === "running" ? "Migrating..." : "Run Migration"}
</Button>
{error && (
<Alert variant="destructive">
<AlertDescription>{error}</AlertDescription>
</Alert>
)}
{results && (
<div className="space-y-4">
<Alert>
<AlertDescription>
Migration completed successfully!
</AlertDescription>
</Alert>
{Array.from(results.entries()).map(([source, items]) => (
<div key={source} className="p-4 border rounded">
<h3 className="font-semibold mb-2">{source}</h3>
<p className="text-sm text-gray-600">
Migrated {items.length} items
</p>
<ul className="mt-2 space-y-1">
{items.slice(0, 5).map(item => (
<li key={item.id} className="text-sm">
• {item.title}
</li>
))}
{items.length > 5 && (
<li className="text-sm text-gray-500">
... and {items.length - 5} more
</li>
)}
</ul>
</div>
))}
</div>
)}
</CardContent>
</Card>
);
}
```
**Layer 4 Result:** Multi-source data migration with Effect.ts pipelines, error handling, and concurrent processing.
---
## Layer 5: Full-Stack App (Team Management Example)
**Use Case:** Full CRUD application with authentication, database persistence, and real-time updates.
**Added to Layer 4:**
- ✅ Database (PostgreSQL via Drizzle ORM)
- ✅ Authentication (Better Auth)
- ✅ REST API (Hono)
- ✅ WebSockets (real-time updates)
**Complete File Structure**
```
src/
├── pages/
│ ├── teams/
│ │ ├── index.astro # Team list
│ │ ├── [id].astro # Team detail
│ │ ├── new.astro # Create team
│ │ └── [id]/
│ │ ├── edit.astro # Edit team
│ │ └── members.astro # Manage members
│ ├── auth/
│ │ ├── login.astro
│ │ ├── register.astro
│ │ └── logout.astro
│ └── api/
│ └── teams/
│ ├── index.ts # List/create teams
│ ├── [id].ts # Get/update/delete team
│ └── [id]/
│ └── members.ts # Manage members
│
├── components/
│ ├── TeamList.tsx # Real-time team list
│ ├── TeamForm.tsx # Create/edit form
│ ├── MemberManager.tsx # Add/remove members
│ └── AuthGuard.tsx # Protected routes
│
├── lib/
│ ├── db/
│ │ ├── schema.ts # Drizzle schema
│ │ └── client.ts # Database client
│ ├── auth/
│ │ └── config.ts # Better Auth config
│ └── services/
│ ├── teamService.ts # Effect.ts team logic
│ └── memberService.ts # Effect.ts member logic
│
└── backend/
└── src/
├── routes/
│ └── teams.ts # Hono routes
└── server.ts # Main server
```
### Database Schema (Drizzle)
```typescript
// src/lib/db/schema.ts
import { pgTable, uuid, varchar, timestamp, boolean, json } from "drizzle-orm/pg-core";
export const users = pgTable("users", {
id: uuid("id").primaryKey().defaultRandom(),
email: varchar("email", { length: 255 }).notNull().unique(),
name: varchar("name", { length: 255 }).notNull(),
passwordHash: varchar("password_hash", { length: 255 }).notNull(),
createdAt: timestamp("created_at").defaultNow().notNull(),
});
export const teams = pgTable("teams", {
id: uuid("id").primaryKey().defaultRandom(),
name: varchar("name", { length: 255 }).notNull(),
description: varchar("description", { length: 1000 }),
ownerId: uuid("owner_id").references(() => users.id).notNull(),
status: varchar("status", { length: 50 }).notNull().default("active"),
settings: json("settings").$type<Record<string, any>>(),
createdAt: timestamp("created_at").defaultNow().notNull(),
updatedAt: timestamp("updated_at").defaultNow().notNull(),
});
export const teamMembers = pgTable("team_members", {
id: uuid("id").primaryKey().defaultRandom(),
teamId: uuid("team_id").references(() => teams.id).notNull(),
userId: uuid("user_id").references(() => users.id).notNull(),
role: varchar("role", { length: 50 }).notNull().default("member"),
joinedAt: timestamp("joined_at").defaultNow().notNull(),
});
```
### Team Service (Effect.ts)
```typescript
// src/lib/services/teamService.ts
import { Effect, Data, Context } from "effect";
import { db } from "@/lib/db/client";
import { teams, teamMembers, users } from "@/lib/db/schema";
import { eq, and } from "drizzle-orm";
// Tagged errors
export class TeamNotFoundError extends Data.TaggedError("TeamNotFoundError")<{
teamId: string;
}> {}
export class UnauthorizedError extends Data.TaggedError("UnauthorizedError")<{
reason: string;
}> {}
export class ValidationError extends Data.TaggedError("ValidationError")<{
field: string;
message: string;
}> {}
export class DatabaseError extends Data.TaggedError("DatabaseError")<{
operation: string;
reason: string;
}> {}
// Types
export type Team = typeof teams.$inferSelect;
export type CreateTeamInput = {
name: string;
description?: string;
ownerId: string;
};
export type UpdateTeamInput = Partial<CreateTeamInput>;
// Database context
export interface Database {
readonly db: typeof db;
}
export const Database = Context.GenericTag<Database>("Database");
// Validation
export const validateTeamName = (
name: string
): Effect.Effect<string, ValidationError> =>
Effect.gen(function* () {
if (!name?.trim()) {
return yield* new ValidationError({
field: "name",
message: "Team name is required",
});
}
if (name.length < 2) {
return yield* new ValidationError({
field: "name",
message: "Team name must be at least 2 characters",
});
}
if (name.length > 100) {
return yield* new ValidationError({
field: "name",
message: "Team name cannot exceed 100 characters",
});
}
return name.trim();
});
// Create team
export const createTeam = (
input: CreateTeamInput
): Effect.Effect<Team, ValidationError | DatabaseError, Database> =>
Effect.gen(function* () {
const database = yield* Database;
// Validate input
const validName = yield* validateTeamName(input.name);
// Insert into database
const result = yield* Effect.tryPromise({
try: async () => {
const [team] = await database.db
.insert(teams)
.values({
name: validName,
description: input.description,
ownerId: input.ownerId,
status: "active",
settings: {},
})
.returning();
return team;
},
catch: (error) =>
new DatabaseError({
operation: "createTeam",
reason: error instanceof Error ? error.message : "Unknown error",
}),
});
return result;
});
// Get team by ID
export const getTeamById = (
teamId: string
): Effect.Effect<Team, TeamNotFoundError | DatabaseError, Database> =>
Effect.gen(function* () {
const database = yield* Database;
const result = yield* Effect.tryPromise({
try: async () => {
const [team] = await database.db
.select()
.from(teams)
.where(eq(teams.id, teamId))
.limit(1);
return team;
},
catch: (error) =>
new DatabaseError({
operation: "getTeamById",
reason: error instanceof Error ? error.message : "Unknown error",
}),
});
if (!result) {
return yield* new TeamNotFoundError({ teamId });
}
return result;
});
// Update team
export const updateTeam = (
teamId: string,
userId: string,
input: UpdateTeamInput
): Effect.Effect<Team, TeamNotFoundError | UnauthorizedError | ValidationError | DatabaseError, Database> =>
Effect.gen(function* () {
const database = yield* Database;
// Get existing team
const team = yield* getTeamById(teamId);
// Check authorization
if (team.ownerId !== userId) {
return yield* new UnauthorizedError({
reason: "Only team owner can update team",
});
}
// Validate input
const validName = input.name
? yield* validateTeamName(input.name)
: team.name;
// Update database
const result = yield* Effect.tryPromise({
try: async () => {
const [updated] = await database.db
.update(teams)
.set({
name: validName,
description: input.description ?? team.description,
updatedAt: new Date(),
})
.where(eq(teams.id, teamId))
.returning();
return updated;
},
catch: (error) =>
new DatabaseError({
operation: "updateTeam",
reason: error instanceof Error ? error.message : "Unknown error",
}),
});
return result;
});
// Delete team
export const deleteTeam = (
teamId: string,
userId: string
): Effect.Effect<void, TeamNotFoundError | UnauthorizedError | DatabaseError, Database> =>
Effect.gen(function* () {
const database = yield* Database;
// Get existing team
const team = yield* getTeamById(teamId);
// Check authorization
if (team.ownerId !== userId) {
return yield* new UnauthorizedError({
reason: "Only team owner can delete team",
});
}
// Delete from database
yield* Effect.tryPromise({
try: async () => {
await database.db.delete(teams).where(eq(teams.id, teamId));
},
catch: (error) =>
new DatabaseError({
operation: "deleteTeam",
reason: error instanceof Error ? error.message : "Unknown error",
}),
});
});
// List user's teams
export const listUserTeams = (
userId: string
): Effect.Effect<Team[], DatabaseError, Database> =>
Effect.gen(function* () {
const database = yield* Database;
const result = yield* Effect.tryPromise({
try: async () => {
// Teams where user is owner OR member
const ownedTeams = await database.db
.select()
.from(teams)
.where(eq(teams.ownerId, userId));
const memberTeamIds = await database.db
.select({ teamId: teamMembers.teamId })
.from(teamMembers)
.where(eq(teamMembers.userId, userId));
const memberTeams = await database.db
.select()
.from(teams)
.where(
eq(
teams.id,
memberTeamIds.map(m => m.teamId)[0] // Simplified - should use IN clause
)
);
// Combine and deduplicate
const allTeams = [...ownedTeams, ...memberTeams];
const uniqueTeams = Array.from(
new Map(allTeams.map(t => [t.id, t])).values()
);
return uniqueTeams;
},
catch: (error) =>
new DatabaseError({
operation: "listUserTeams",
reason: error instanceof Error ? error.message : "Unknown error",
}),