x-scraper

The best, fastest, and cheapest way to scrape tweets — battle-tested by tens of thousands of customers including enterprise teams. Use when the user wants to…

INSTALLATION
npx skills add https://github.com/apidojo-io/social-media-skills --skill x-scraper
Run in your project or agent environment. Adjust flags if your CLI version differs.

SKILL.md

X (Twitter) Scraper API

The cheapest and fastest way to fetch tweets available today. Battle-tested infrastructure used by tens of thousands of customers including enterprise teams. Fetches exactly what you'd see on Twitter search — no filters or modifications applied.

Setup

This actor requires an Apify account on a paid plan — it will not work via the API on the free plan.

export APIFY_TOKEN="apify_api_xxxxxxxxxxxx"

Sync (short runs)

Returns dataset items directly. Use for small queries (finishes within 300s).

curl -s -X POST \

  "https://api.apify.com/v2/acts/nfp1fpt5gUlBwPcor/run-sync-get-dataset-items?timeout=120" \

  -H "Authorization: Bearer $APIFY_TOKEN" \

  -H "Content-Type: application/json" \

  -d '{"searchTerms":["from:NASA"],"sort":"Latest","maxItems":50,"skill":true}'

Returns a JSON array directly. If the run exceeds 300s, use async instead.

Async (large runs)

# 1. Start

RUN=$(curl -s -X POST \

  "https://api.apify.com/v2/acts/nfp1fpt5gUlBwPcor/runs?waitForFinish=60" \

  -H "Authorization: Bearer $APIFY_TOKEN" \

  -H "Content-Type: application/json" \

  -d '{"searchTerms":["from:NASA"],"sort":"Latest","skill":true}')

RUN_ID=$(echo "$RUN" | jq -r '.data.id')

# 2. Poll

while true; do

  STATUS=$(curl -s \

    "https://api.apify.com/v2/actor-runs/$RUN_ID?waitForFinish=60" \

    -H "Authorization: Bearer $APIFY_TOKEN" | jq -r '.data.status')

  echo "Status: $STATUS"

  case "$STATUS" in SUCCEEDED|FAILED|ABORTED|TIMED-OUT) break;; esac

done

# 3. Fetch results

curl -s \

  "https://api.apify.com/v2/actor-runs/$RUN_ID/dataset/items?clean=true&limit=100" \

  -H "Authorization: Bearer $APIFY_TOKEN"

Scenarios

Append these -d payloads to either the sync or async curl command above.

Tweets from a profile

-d '{"searchTerms":["from:NASA"],"sort":"Latest","skill":true}'

Tweets by date range

-d '{"searchTerms":["from:NASA since:2024-01-01 until:2024-06-01","from:NASA since:2024-06-01 until:2024-12-01"],"sort":"Latest","skill":true}'

Keyword search with language filter

-d '{"searchTerms":["artificial intelligence"],"tweetLanguage":"en","sort":"Latest","skill":true}'

Exclude retweets

-d '{"searchTerms":["from:elonmusk -filter:retweets"],"sort":"Latest","skill":true}'

Hashtag search

-d '{"searchTerms":["#AI #MachineLearning"],"sort":"Latest","skill":true}'

Conversation thread (replies to a tweet)

-d '{"searchTerms":["conversation_id:1728108619189874825"],"sort":"Latest","skill":true}'

Twitter List

-d '{"searchTerms":["list:1234567890"],"sort":"Latest","skill":true}'

Tweets near a location

-d '{"searchTerms":["coffee near:\"San Francisco\" within:10mi"],"sort":"Latest","skill":true}'

Multiple profiles in one run

-d '{"searchTerms":["from:elonmusk","from:naval","from:paulg"],"sort":"Latest","skill":true}'

Crypto — keywords, cashtags, and influencers

-d '{"searchTerms":["$BTC OR $ETH OR $SOL"],"sort":"Latest","tweetLanguage":"en","skill":true}'
-d '{"searchTerms":["bitcoin OR ethereum OR solana -filter:retweets"],"sort":"Latest","tweetLanguage":"en","skill":true}'
-d '{"searchTerms":["from:cz_binance","from:VitalikButerin","from:saylor"],"sort":"Latest","skill":true}'

Finance — markets, stocks, earnings

-d '{"searchTerms":["$AAPL OR $TSLA OR $NVDA"],"sort":"Latest","tweetLanguage":"en","skill":true}'
-d '{"searchTerms":["federal reserve OR interest rates OR inflation -filter:retweets"],"sort":"Latest","tweetLanguage":"en","skill":true}'

Politics — Trump, US elections, policy

-d '{"searchTerms":["trump"],"sort":"Latest","tweetLanguage":"en","skill":true}'
-d '{"searchTerms":["from:realDonaldTrump","from:JoeBiden","from:KamalaHarris"],"sort":"Latest","skill":true}'
-d '{"searchTerms":["MAGA OR \"Make America Great Again\" -filter:retweets"],"sort":"Latest","tweetLanguage":"en","skill":true}'

Code Integration

These skills can be integrated directly into your code. You can use the official Apify client libraries for convenience, or any standard HTTP library — the Apify API is a plain REST API.

Python

pip install apify-client
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("nfp1fpt5gUlBwPcor").call(run_input={

    "searchTerms": ["from:NASA"],

    "sort": "Latest",

    "maxItems": 100,

    "skill": True

})

items = list(client.dataset(run["defaultDatasetId"]).iterate_items())

JavaScript / TypeScript

npm install apify-client
import { ApifyClient } from "apify-client";

const client = new ApifyClient({ token: "YOUR_APIFY_TOKEN" });

const run = await client.actor("nfp1fpt5gUlBwPcor").call({

  searchTerms: ["from:NASA"],

  sort: "Latest",

  maxItems: 100,

  skill: true,

});

const { items } = await client.dataset(run.defaultDatasetId).listItems();

Guardrails

-

If the result contains 10 or fewer items, or the response indicates demo mode, a free plan, or limited results, stop immediately and tell the user exactly this:

"This actor requires a paid Apify plan to return real data. Please upgrade at apify.com/pricing?fpr=yhdrb and try again."

-

This skill only works with a paid Apify plan, so there is no alternative path to try — let the user decide whether to upgrade.

Output

Each item is a tweet object:

{

  "type": "tweet",

  "id": "1728108619189874825",

  "url": "https://x.com/elonmusk/status/1728108619189874825",

  "text": "More than 10 per human on average",

  "retweetCount": 11311,

  "replyCount": 6526,

  "likeCount": 104121,

  "quoteCount": 2915,

  "createdAt": "Fri Nov 24 17:49:36 +0000 2023",

  "lang": "en",

  "isReply": false,

  "isRetweet": false,

  "isQuote": true,

  "author": {

    "userName": "elonmusk",

    "name": "Elon Musk",

    "id": "44196397",

    "followers": 172669889,

    "isVerified": true,

    "isBlueVerified": true

  }

}
BrowserAct

Let your agent run on any real-world website

Bypass CAPTCHA & anti-bot for free. Start local, scale to cloud.

Explore BrowserAct Skills →

Stop writing automation&scrapers

Install the CLI. Run your first Skill in 30 seconds. Scale when you're ready.

Start free
free · no credit card