Publishing setup

# RankGoat auto-publish - webhook integration

Build ONE HTTPS endpoint in my project that receives RankGoat's signed JSON POSTs and publishes blog posts to my site. Detect my framework / CMS and hosting from the codebase (ask me if it's unclear), implement the handler, then tell me what URL to paste into RankGoat (Settings -> Publishing) and how to test it. Read the shared secret from env var RANKGOAT_SECRET (never hardcode it).

## Security - verify BEFORE parsing the body
- expected = "sha256=" + hex(HMAC-SHA256(rawRequestBodyBytes, RANKGOAT_SECRET))
- Constant-time compare to the X-RankGoat-Signature header. On mismatch respond 401 and stop.
- Reject with 401 if X-RankGoat-Timestamp is more than 5 minutes old.
- The HMAC must cover the EXACT raw request bytes. Capture the raw body; do not let the framework parse then re-serialize the JSON first.
- Respond with JSON within 30s. On any non-2xx or timeout RankGoat retries at +5min and +1hr, then emails me.

Headers on every request:
```
X-RankGoat-Event:     ping | media.upload | post.publish | post.update | post.delete
X-RankGoat-Timestamp: 1715900000          // unix seconds; reject if >5 min old
X-RankGoat-Signature: sha256=<hex>         // HMAC-SHA256(rawBody, RANKGOAT_SECRET)
```

## Events
Every event is a POST to the same URL; branch on the `event` field.

### ping
Connectivity test. No body beyond event/timestamp. Respond 200 with the contract version you implement:
```json
{ "ok": true, "version": 2 }
```
**version** tells RankGoat which events your endpoint supports. Current is **2** (adds post.delete). Omit it (or send 1) and we treat you as the original contract: we won't send newer events (a member archiving a live post is told to update the integration instead). Bump it once you handle the new events. Tip: always respond 200 to events you don't recognise so future additions never break you.

### media.upload
Sent before any post with images. Store each file and return its public URL; RankGoat then rewrites the article to your URLs. Files you don't return are dropped (RankGoat won't host them).
Request body:
- `event` (string, required): "media.upload".
- `timestamp` (integer, required): Unix seconds.
- `post_slug` (string, required): Slug of the post these images belong to.
- `media` (array, required): One entry per image.
- `media[].filename` (string, required): Use as the key in your response.
- `media[].content_type` (string, required): image/webp, image/png, or image/jpeg.
- `media[].data_base64` (string, required): Base64-encoded file bytes.
Example request:
```json
{
  "event": "media.upload",
  "timestamp": 1715900000,
  "post_slug": "speed-up-your-static-site",
  "media": [
    { "filename": "post-42.webp", "content_type": "image/webp", "data_base64": "UklGR..." }
  ]
}
```
Response body:
- `media` (array, required): One entry per file you stored. Omit a file to drop it.
- `media[].filename` (string, required): Echo the filename we sent.
- `media[].url` (string, required): Public URL where you host it.
Example response:
```json
{
  "media": [
    { "filename": "post-42.webp", "url": "https://yoursite.com/media/post-42.webp" }
  ]
}
```

### post.publish
Create the post and return its live URL. body_html and hero_image_url already point at the media URLs you returned.
Top-level fields:
- `event` (string, required): "post.publish", "post.update", or "post.delete".
- `timestamp` (integer, required): Unix seconds. Reject if over 5 minutes old.
- `post` (object, required): The article (fields below).
- `outbound_links` (array, may be empty): Member links to embed. Keep each exactly; never add nofollow.
- `related_articles` (array, may be empty): Links to your other posts, same topic cluster first. Strongly recommended to render (a "Related articles" list is the usual pattern): these internal links are how Google discovers and crawls the rest of your content.
post object:
- `id` (integer, required): Our post id.
- `site_id` (integer, required): Our site id.
- `title` (string, required)
- `slug` (string, required): URL-safe, stable. Match on this for post.update.
- `meta_description` (string, may be empty): Use as the excerpt / meta description.
- `body_html` (string, required): Full HTML. Images already point at your media URLs.
- `hero_image_url` (string | null, nullable): Same image as the first <figure> in body_html (for a featured-image / og:image slot). null when there is no hero.
- `json_ld` (object, required): schema.org BlogPosting + FAQPage. Inject verbatim into <head>.
- `language` (string, required): BCP-47, e.g. en.
- `alternates` (array, may be empty): Translations [{ lang, url }]. Empty today.
- `word_count` (integer | null, nullable)
- `category` (string | null, nullable): Format label (How-to, Comparison, ...).
- `tags` (string[], may be empty): Post tags.
outbound_links[]:
- `target_domain` (string, required): e.g. techreview.io.
- `anchor` (string, required): Anchor text. Keep exactly.
- `href` (string, required): https://{target_domain}.
related_articles[]:
- `title` (string, required)
- `url` (string, required): Full URL to your post.
Example request:
```json
{
  "event": "post.publish",
  "timestamp": 1715900000,
  "post": {
    "id": 42,
    "site_id": 7,
    "title": "Speed Up Your Static Site",
    "slug": "speed-up-your-static-site",
    "meta_description": "Five fixes that cut load time.",
    "body_html": "<figure><img src='https://yoursite.com/media/post-42.webp'></figure><p>...</p>",
    "hero_image_url": "https://yoursite.com/media/post-42.webp",
    "json_ld": { "@context": "https://schema.org", "@graph": [ { "@type": "BlogPosting" }, { "@type": "FAQPage" } ] },
    "language": "en",
    "alternates": [],
    "word_count": 1180,
    "category": "How-to",
    "tags": ["static sites", "performance"]
  },
  "outbound_links": [
    { "target_domain": "techreview.io", "anchor": "build pipelines", "href": "https://techreview.io" }
  ],
  "related_articles": [
    { "title": "Lazy-Loading Images", "url": "https://yoursite.com/blog/lazy-loading-images" }
  ]
}
```
Example response:
```json
{ "published_url": "https://yoursite.com/blog/speed-up-your-static-site" }
```

### post.update
Sent when a member edits a live post. Identical payload to post.publish. Match on post.slug and update the existing post in place (do NOT create a duplicate or change its URL). If the slug is unknown, create it. Respond with the live published_url.

### post.delete
Sent when a member archives a live post - remove it from your site. Minimal payload: `{ "event": "post.delete", "timestamp", "post": { "id", "slug" } }` (no body_html or links). Delete or unpublish the post matched by post.slug and respond 200 (any JSON, e.g. `{"ok":true}`). Idempotent: if the post is already gone, still respond 200. A restore later re-sends the article as a normal post.update (which creates it again if the slug is unknown), so trashing rather than hard-deleting is fine.

## Rendering rules (get these right)
- Hero image, no duplicates: the hero is ALREADY the first <figure> in body_html. hero_image_url is that exact same image, for a featured-image / og:image slot only - not a second image to insert. If you set a featured image and your theme renders it above the content, strip that leading <figure> from body_html first, or skip the featured image. Never add hero_image_url as an extra <img> in the body.
- json_ld: inject verbatim into the page <head> as <script type="application/ld+json">...</script> (schema.org BlogPosting + FAQPage). The visible FAQ is already in body_html.
- Self-canonical: each post's canonical points at its own URL.
- alternates (hreflang): when non-empty (list of {lang, url}), emit reciprocal <link rel="alternate" hreflang="..."> tags plus an x-default. It is [] today.
- outbound_links: keep every anchor and href exactly as sent; never add rel="nofollow". RankGoat checks these daily and pauses links that change.
- related_articles: links to my other posts ({title, url}). Render as a "Related articles" block (dofollow) - strongly recommended, these internal links are how Google discovers and crawls the rest of the site's content. Skip only when the array is empty.
- Idempotent on slug: a repeated slug updates the existing post, never creates a duplicate.

## Contract versions
Report the contract version you implement from your ping response: { "ok": true, "version": 2 }. RankGoat uses it to know which events you can receive. Current version: 2.
- v1 (original): ping, media.upload, post.publish, post.update.
- v2 adds post.delete (remove a post by slug when a member archives it); ping now returns version.

Upgrading v1 -> v2: add a post.delete handler (delete or unpublish by slug, respond 200, idempotent), then return "version": 2 from your ping and re-test the connection. Until your ping reports version >= 2, RankGoat won't send post.delete - an archived post stays on your site and the member is told to remove it manually. Tip: respond 200 to any event you don't recognise so future versions never break you.

When done, tell me the endpoint URL to paste into RankGoat, confirm the secret goes in RANKGOAT_SECRET, and how to test it locally.

Any other stack: host one HTTPS endpoint that accepts our signed JSON POSTs and branches on the event field - ping, media.upload, post.publish, post.update, post.delete. Full schema and examples below. New here? Build to the current contract version.

Every request

X-RankGoat-Event:     ping | media.upload | post.publish | post.update | post.delete
X-RankGoat-Timestamp: 1715900000          // unix seconds; reject if >5 min old
X-RankGoat-Signature: sha256=<hex>         // HMAC-SHA256(rawBody, RANKGOAT_SECRET)

Verify the signature against the raw body bytes (constant-time) before parsing. Respond with JSON within 30s. On any non-2xx or timeout we retry at +5 min and +1 hr, then email you.

Every request reaches you from one fixed IP: 178.105.170.30. If your endpoint sits behind Cloudflare, a WAF, or a firewall, allowlist it - see Blocked by a firewall or Cloudflare below.

Every field below is always present. Nullable = can be null; may be empty = empty string or array. Handle both.

{ "ok": true, "version": 2 }

{
  "event": "media.upload",
  "timestamp": 1715900000,
  "post_slug": "speed-up-your-static-site",
  "media": [
    { "filename": "post-42.webp", "content_type": "image/webp", "data_base64": "UklGR..." }
  ]
}

{
  "media": [
    { "filename": "post-42.webp", "url": "https://yoursite.com/media/post-42.webp" }
  ]
}

{
  "event": "post.publish",
  "timestamp": 1715900000,
  "post": {
    "id": 42,
    "site_id": 7,
    "title": "Speed Up Your Static Site",
    "slug": "speed-up-your-static-site",
    "meta_description": "Five fixes that cut load time.",
    "body_html": "<figure><img src='https://yoursite.com/media/post-42.webp'></figure><p>...</p>",
    "hero_image_url": "https://yoursite.com/media/post-42.webp",
    "json_ld": { "@context": "https://schema.org", "@graph": [ { "@type": "BlogPosting" }, { "@type": "FAQPage" } ] },
    "language": "en",
    "alternates": [],
    "word_count": 1180,
    "category": "How-to",
    "tags": ["static sites", "performance"]
  },
  "outbound_links": [
    { "target_domain": "techreview.io", "anchor": "build pipelines", "href": "https://techreview.io" }
  ],
  "related_articles": [
    { "title": "Lazy-Loading Images", "url": "https://yoursite.com/blog/lazy-loading-images" }
  ]
}

{ "published_url": "https://yoursite.com/blog/speed-up-your-static-site" }

{ "published_url": "https://yoursite.com/blog/speed-up-your-static-site" }

{
  "event": "post.delete",
  "timestamp": 1715900000,
  "post": { "id": 123, "slug": "my-post-slug" }
}

{ "ok": true }

Languages

Each page is self-canonical (canonical points at its own URL). When alternates is non-empty, emit reciprocal <link rel="alternate" hreflang="..."> tags plus an x-default. It's [] until translations exist.

Handler

import express from "express";
import crypto from "node:crypto";

const app = express();
app.use(express.raw({ type: "application/json" })); // raw body so HMAC matches

const SECRET = process.env.RANKGOAT_SECRET;

app.post("/api/rankgoat-publish", async (req, res) => {
  const sig = req.get("X-RankGoat-Signature") || "";
  const expected = "sha256=" + crypto.createHmac("sha256", SECRET).update(req.body).digest("hex");
  if (!crypto.timingSafeEqual(Buffer.from(sig), Buffer.from(expected))) {
    return res.status(401).json({ error: "bad signature" });
  }

  const payload = JSON.parse(req.body.toString("utf8"));
  if (payload.event === "ping") return res.json({ ok: true, version: 2 }); // version = contract you implement

  if (payload.event === "media.upload") {
    const media = await Promise.all(payload.media.map(async (m) => ({
      filename: m.filename,
      url: await saveToStorage(m.filename, Buffer.from(m.data_base64, "base64")),
    })));
    return res.json({ media });
  }

  // post.publish and post.update share this shape; upsert on slug.
  if (payload.event === "post.publish" || payload.event === "post.update") {
    const post = payload.post;
    await db.posts.upsert({ slug: post.slug }, {
      title: post.title,
      body_html: post.body_html,
      meta_description: post.meta_description,
      json_ld: JSON.stringify(post.json_ld), // print in <head> as <script type="application/ld+json">
    });
    return res.json({ published_url: `https://yoursite.com/blog/${post.slug}` });
  }

  // post.delete: member archived the post - remove it by slug. Idempotent.
  if (payload.event === "post.delete") {
    await db.posts.deleteBySlug(payload.post.slug); // no-op if already gone
    return res.json({ ok: true });
  }

  res.json({ ok: true }); // ignore events you don't recognise (forward-compatible)
});

app.listen(8080);

import hmac, hashlib, os, json, base64
from flask import Flask, request, jsonify

app = Flask(__name__)
SECRET = os.environ["RANKGOAT_SECRET"].encode()

@app.post("/api/rankgoat-publish")
def rankgoat_publish():
    body = request.get_data()
    sig  = request.headers.get("X-RankGoat-Signature", "")
    expected = "sha256=" + hmac.new(SECRET, body, hashlib.sha256).hexdigest()
    if not hmac.compare_digest(expected, sig):
        return jsonify(error="bad signature"), 401

    payload = json.loads(body)
    if payload["event"] == "ping":
        return jsonify(ok=True, version=2)  # version = contract you implement

    if payload["event"] == "media.upload":
        media = [
            {"filename": m["filename"], "url": save_to_storage(m["filename"], base64.b64decode(m["data_base64"]))}
            for m in payload["media"]
        ]
        return jsonify(media=media)

    # post.delete: member archived the post - remove it by slug. Idempotent.
    if payload["event"] == "post.delete":
        your_cms.delete_by_slug(payload["post"]["slug"])  # no-op if already gone
        return jsonify(ok=True)

    # post.publish and post.update share this shape; upsert on slug.
    post = payload["post"]
    your_cms.upsert_post(
        slug=post["slug"],
        title=post["title"],
        body=post["body_html"],
        meta=post["meta_description"],
        json_ld=post["json_ld"],
    )
    return jsonify(published_url=f"https://yoursite.com/blog/{post['slug']}")

Ghost can't receive webhooks - point the Node handler at Ghost's Admin API.

Rules

• Verify the signature against raw body bytes (constant-time), before parsing. Reject timestamps over 5 min old.
• Make publishing idempotent on slug - a repeated slug updates, never duplicates.
• Never change embedded links (anchor, href) or add nofollow. Checked daily.

Appendix

Blocked by a firewall or Cloudflare? (403)

If you only see some events (for example post.publish but never media.upload), or a published post arrives with no image, your edge is most likely blocking us. media.upload carries the image and has a larger body, so a WAF or bot rule often blocks it while letting smaller events through. The tell is a 403 whose body is an HTML challenge page (not your app's JSON) - that means the request never reached your server.

Fix: allowlist our publishing IP 178.105.170.30 so our requests skip your bot/WAF rules:

• Cloudflare: Security → WAF → Custom rules (or Tools → IP Access Rules): add a Skip / Allow rule for IP 178.105.170.30 on your endpoint's path. Also exclude it from Bot Fight Mode.
• nginx / Apache / firewall: allow 178.105.170.30 to your endpoint path, and make sure your request body size limit (e.g. nginx client_max_body_size) accepts a few MB - images are sent inline.
• Vercel / Netlify / serverless: add the IP to any firewall/bot-protection allowlist, and confirm your function's body-size limit isn't rejecting the image.

After allowlisting, open the post in Posts and use Re-send to deliver it again. A 413 means a body-size limit (raise it); a 401 means the signature/secret - re-check your secret in Settings → Publishing.

Signature rejected (401)

The secret your endpoint verifies with doesn't match the one we sign with. Copy the secret from Settings → Publishing into your endpoint's environment, redeploy, then run Test connection and re-send the post.

Endpoint returns 404

The webhook URL doesn't match a route on your server. Double-check the full URL (including the path) in Settings → Publishing, and make sure the route accepts POST.

Your endpoint errored (HTTP 5xx)

The request reached your endpoint, but your code threw while handling it - the fix is on your side, and your server or function logs have the stack trace. Common causes: verifying the signature against the parsed body instead of the raw bytes, a failing database write, or an unhandled event type (respond 200 to any event you don't recognise so new event types never break you). Once fixed, re-send the post from Posts.

Endpoint unreachable or timing out

We couldn't connect, or your endpoint took longer than 30 seconds to respond. Make sure it's deployed, publicly reachable over HTTPS (not localhost, not behind auth), and responds fast - acknowledge with 200 first and do slow work afterwards. Then re-send the post.

Replied with non-JSON

Your endpoint must answer 200 with a JSON body, e.g. {"ok":true}. An HTML error page, a redirect, or an empty body counts as a failed delivery. Check what your route actually returns (a proxy or error middleware may be answering instead of your handler), then re-send.

Contract versions

Report the version you implement from your ping: { "ok": true, "version": 2 }. RankGoat uses it to know which events you can receive. Current version: 2.

• v1 (original): ping, media.upload, post.publish, post.update.
• v2 adds post.delete - remove a post by slug when a member archives it; ping now returns version.

Upgrading v1 → v2: add a post.delete handler (delete/unpublish by slug, respond 200, idempotent), then return "version": 2 from ping and re-run Test connection. Until your ping reports v2, we won't send post.delete - archived posts stay on your site and you remove them manually. Tip: respond 200 to any event you don't recognise so future versions never break you.