AI Video Prompts: Direct Runway, Kling & Pika

AI Video Prompts: How to Direct Runway, Kling & Pika Like a Pro

AI video generation is the newest frontier in generative AI. In 2024, tools like Runway Gen-3, Kling, and Pika produced the first genuinely usable AI video clips. By 2026, they're approaching production-ready quality for specific use cases.

But video is significantly harder than images. Each clip is hundreds of individual frames that must remain coherent. Motion must feel natural. Physics must be convincing. And all of this needs to be controlled through text prompts.

This guide teaches you how to prompt AI video generators like a cinematographer, covering camera movement vocabulary, motion descriptors, and platform-specific techniques for Runway, Kling, and Pika.

How Video AI Works

Understanding the technical basics helps you write better prompts.

Two primary modes:

Text-to-video:

• Write a prompt describing the scene
• AI generates video from scratch
• Starting from noise (like image generation)
• More creative freedom, less control

Image-to-video:

• Provide a starting image (AI-generated or real photo)
• Write a prompt describing the motion
• AI animates the still image
• More control, more predictable results

Why video is harder than images:

Temporal coherence:

• Every frame must be consistent with the previous frame
• Characters/objects can't morph or disappear mid-clip
• Lighting and atmosphere must stay stable
• Current AI struggles with this beyond 3-10 seconds

Physics simulation:

• Motion must obey physics (gravity, momentum, inertia)
• AI learned physics implicitly from training videos
• Still produces impossible movements regularly

Camera work:

• AI must understand camera movement (pan, zoom, dolly, etc.)
• Interpret whether motion is subject moving or camera moving
• Maintain framing and composition throughout

Computing cost:

• Video generation is 100-1000× more expensive than images
• Limits iteration speed
• Most platforms charge per-second generated

The field is evolving rapidly. What's impossible today may be trivial in six months.

The Video Prompt Formula

Effective video prompts need more specificity than image prompts.

Core formula:

[Scene description] + [Subject action] + [Camera movement] + [Lighting] + [Mood] + [Technical details] + [Duration context]

Example:

A woman walking through a forest at golden hour, slow motion, camera follows beside her, warm afternoon light, peaceful atmosphere, cinematic, 4K

Component priority for video:

1. Scene (setting and context)
2. Action (what moves, how it moves)
3. Camera movement (static, pan, zoom, dolly, etc.)
4. Subject details (appearance, clothing, expression)
5. Lighting (natural, dramatic, soft, etc.)
6. Mood/atmosphere
7. Technical specs (4K, slow-mo, frame rate)

Key difference from image prompts:

• Action verbs are critical — video needs movement
• Camera language is essential — must specify camera behavior
• Duration cues help (slow, fast, time-lapse, slow motion)

Optimal prompt length:

• Runway Gen-3: 20-50 words
• Kling: 25-60 words
• Pika: 15-40 words

Longer than image prompts because motion needs description.

Camera Movement Vocabulary

These words reliably trigger specific camera behaviors. Master them.

Basic Camera Movements

Static shot:

"Static camera, no movement"
"Fixed camera angle"
"Locked camera"

Effect: Camera doesn't move. Subject movement only.

Pan (horizontal camera rotation):

"Pan left" → Camera rotates left, scene moves right across frame
"Pan right" → Camera rotates right, scene moves left across frame
"Slow pan across the landscape"

Effect: Reveals scene horizontally, like turning your head.

Tilt (vertical camera rotation):

"Tilt up" → Camera rotates upward
"Tilt down" → Camera rotates downward

Effect: Reveals scene vertically, like looking up/down.

Zoom in:

"Zoom in on subject"
"Slow zoom into face"
"Dramatic zoom"

Effect: Magnifies subject, moves closer optically (not physically).

Zoom out:

"Zoom out to reveal"
"Pull back to show full scene"

Effect: Widens view, reveals more context.

Advanced Camera Movements

Dolly (camera moves physically forward/backward):

"Dolly in" → Camera physically moves toward subject
"Dolly out" → Camera physically moves away
"Slow dolly push"

Effect: Similar to zoom but with depth/parallax shift. More cinematic.

Truck (camera moves physically left/right):

"Truck left" → Camera physically slides left
"Truck right" → Camera physically slides right
"Lateral tracking shot"

Effect: Moves parallel to subject, often following action.

Crane/Boom (camera moves vertically):

"Crane up" → Camera rises vertically
"Crane down" → Camera descends
"Ascending crane shot revealing cityscape"

Effect: Dramatic vertical movement, god's-eye view transitions.

Orbit/Arc:

"Orbit around subject"
"Camera circles the character"
"360-degree rotation around object"

Effect: Camera moves in circular path around subject.

Handheld:

"Handheld camera, shaky"
"Documentary style, unstable"
"Realistic handheld movement"

Effect: Subtle shake, organic feeling, increases realism.

Steadicam:

"Smooth steadicam movement"
"Gliding camera following subject"

Effect: Smooth, flowing movement without shake.

Drone/Aerial:

"Aerial drone shot"
"Bird's eye view descending"
"Fly-over establishing shot"

Effect: High altitude perspective, sweeping movements.

First-person POV:

"POV shot, first-person perspective"
"Camera is the character's eyes"

Effect: Viewer sees through character's eyes.

Combination Movements

AI video can handle combined movements:

"Camera orbits around subject while slowly zooming in"
"Dolly in while panning right"
"Crane up and pull back to reveal"

Combinations produce more cinematic results but are less predictable.

Action Descriptors — Making Things Move

Video needs motion. Here's how to describe it.

Subject Movement Verbs

Walking/Running:

"walking slowly toward camera"
"running away from camera"
"strolling through park"
"sprinting across field"

Looking/Turning:

"turns head to look at camera"
"glances over shoulder"
"looks up suddenly"

Hand/Arm movements:

"waves hand"
"reaches out"
"raises arms triumphantly"

Object interaction:

"picks up object"
"places cup on table"
"opens door"

Facial expressions:

"smiles gradually"
"eyes widen in surprise"
"expression shifts from sad to hopeful"

Environmental Motion

Natural elements:

"leaves rustling in wind"
"water flowing over rocks"
"clouds drifting across sky"
"rain falling heavily"
"fire flickering"

Atmospheric effects:

"fog rolling in"
"dust particles floating in light beam"
"smoke rising slowly"

Background activity:

"cars passing in background"
"people walking by out of focus"
"birds flying across frame"

Speed and Tempo Descriptors

"slow motion" → Captures action at reduced speed
"time-lapse" → Accelerated passage of time
"normal speed"
"fast motion, accelerated"
"frame-by-frame, stuttering motion"

Duration cues:

"quick pan" vs. "slow pan"
"sudden zoom" vs. "gradual zoom"
"gentle movement" vs. "rapid movement"

Runway Gen-3 Prompting

Runway is the industry leader in AI video, known for Gen-3's photorealism and motion quality.

Text-to-Video vs. Image-to-Video Modes

Text-to-video:

• More creative freedom
• Less predictable
• Best for abstract concepts, motion graphics, effects

Image-to-video (recommended):

• Start with AI-generated or uploaded image
• Describe motion to apply
• Much more controllable
• Professional results

Workflow for best results:

1. Generate perfect still image (Midjourney, DALL-E, SD)
2. Upload to Runway Gen-3
3. Write motion prompt
4. Generate 5-10 second clip

What Runway Handles Well

Strengths:

• Smooth, realistic motion
• Photorealistic quality
• Natural physics (most of the time)
• Camera movements
• Environmental effects (water, smoke, wind)

Weaknesses:

• Fast action (motion blur, artifacts)
• Complex human movements (dancing, sports)
• Facial expressions (still limited)
• Long durations (quality degrades after 10 seconds)
• Text appearing in video

Motion Brush Technique

Runway Gen-3 has "Motion Brush"—lets you paint which parts of the image should move and in what direction.

How to use:

1. Upload image
2. Select Motion Brush tool
3. Paint over areas you want to move
4. Draw arrow indicating direction
5. Set motion intensity (subtle to strong)
6. Generate

Use cases:

• Make specific elements move while others stay static
• Direct hair, clothing, flags to blow in specific direction
• Control water flow direction
• Animate specific objects while keeping background still

Pro tip: Combine Motion Brush with text prompts for maximum control.

10 Example Runway Prompts with Expected Results

1. Portrait to Subtle Animation

Image: Close-up portrait
Prompt: Gentle breeze moving hair, subject looks slightly to the right, soft smile forms, natural eye blinks, golden hour lighting

Expected: Subtle, realistic portrait animation with natural micro-movements

2. Nature Scene

Image: Forest landscape
Prompt: Camera slowly dollies forward through forest, leaves gently rustling, dappled sunlight shifts, peaceful atmosphere

Expected: Smooth forward movement with environmental details animated

3. Product Reveal

Image: Product on table
Prompt: Camera orbits around product, dramatic lighting, slow rotation revealing all angles, professional commercial style

Expected: 360-degree product showcase, studio-quality lighting

4. Cityscape Time-Lapse

Image: City skyline at dusk
Prompt: Time-lapse transition from dusk to night, lights turn on in buildings, clouds move quickly, traffic streams below

Expected: Accelerated day-to-night transition with urban activity

5. Character Walking

Image: Person standing in street
Prompt: Subject walks toward camera, confident stride, coat billowing slightly, camera slowly dollies back, cinematic

Expected: Natural walking motion with camera tracking backward

6. Abstract Motion Graphics

Text-to-video (no image)
Prompt: Abstract colorful liquid shapes morphing and flowing, smooth transitions, vibrant gradients, mesmerizing motion

Expected: Fluid abstract animation, background/transition material

7. Wildlife Close-Up

Image: Bird on branch
Prompt: Bird turns head, looks at camera, ruffles feathers, subtle branch sway, natural documentary style

Expected: Realistic wildlife micro-movements

8. Establishing Shot

Image: Mountain landscape
Prompt: Aerial drone shot, camera flies forward and up, revealing vast landscape, epic scale, golden hour lighting

Expected: Cinematic establishing shot with altitude and forward movement

9. Food Commercial

Image: Plated dish
Prompt: Slow zoom into food, steam rising, drizzle of sauce in slow motion, professional food photography lighting

Expected: Appetizing commercial-style food shot

10. Sci-Fi Scene

Image: Futuristic city
Prompt: Camera pans across neon-lit city, flying cars pass by, holographic advertisements flicker, rain-slicked streets reflect lights, Blade Runner aesthetic

Expected: Cyberpunk atmosphere with motion and depth

Common Failure Modes and Fixes

Problem: Subject morphs or distorts

Fix: Use image-to-video mode, simpler motion prompts, shorter duration

Problem: Camera movement too aggressive

Fix: Add "slow" or "gentle" to camera movement descriptors

Problem: Motion feels floaty/unrealistic

Fix: Add "natural physics" or "realistic motion" to prompt

Problem: Multiple subjects move wrong

Fix: Use Motion Brush to isolate which parts move

Kling AI Prompting

Kling (Chinese AI video platform) is known for longer coherent sequences—up to 10 seconds of stable motion.

Kling's Strengths

What Kling does better:

• Longer clips with maintained coherence (10 sec vs. 5 sec typical)
• Better at complex scenes
• Strong with Asian aesthetics and subjects
• Good motion consistency

What Kling struggles with:

• Western photorealism (trained more on Asian datasets)
• Extreme closeups
• Very fast action

When to use Kling:

• Need longer clips
• Asian subjects/settings
• When Runway produces too much distortion
• Budget-conscious (often cheaper per second)

Prompt Structure for Kling

Kling responds well to structured, detailed prompts:

[Detailed scene description] + [Subject and action] + [Camera movement] + [Duration/pace] + [Lighting/mood] + [Style reference]

Example:

A serene Japanese garden with koi pond, woman in kimono walks slowly along stone path, camera tracks beside her, cherry blossoms falling gently, soft afternoon light, peaceful atmosphere, cinematic travel video style

8 Example Kling Prompts

1. Urban Street Scene

Busy Tokyo street at night, neon signs reflected on wet pavement, people walking with umbrellas, camera slowly pans right, rain falling, vibrant colors, cyberpunk atmosphere

2. Nature Documentary

Macro shot of butterfly landing on flower, wings slowly open and close, gentle breeze, bokeh background, natural lighting, wildlife documentary style

3. Architectural Flythrough

Modern minimalist house interior, camera glides through rooms, sunlight streaming through windows, clean aesthetic, architectural visualization, smooth steadicam movement

4. Fantasy Landscape

Mystical forest with glowing mushrooms, camera moves forward along path, magical particles floating in air, ethereal lighting, fantasy concept art style

5. Action Sports

Skateboarder performing trick, slow motion capture, camera follows action, urban setting, golden hour light, youth culture aesthetic

6. Food Preparation

Close-up of chef's hands chopping vegetables, smooth knife motion, ingredients on wooden cutting board, natural kitchen lighting, cooking show style

7. Fashion/Portrait

Model walking down runway, confident stride, camera tracks forward movement, studio lighting, high fashion aesthetic, professional production

8. Atmospheric Landscape

Misty mountain valley at sunrise, fog slowly clearing, camera slowly rises to reveal vista, dramatic lighting breaking through clouds, epic nature cinematography

Pika Prompting

Pika focuses on fast, social-media-optimized video generation with unique "Pikaffects."

Pika's Unique Features

Speed:

• Fastest generation times (often under 30 seconds)
• Great for rapid iteration
• Lower cost per clip

Pikaffects:

• Preset effects you can apply: "inflate," "explode," "melt," "crumble," "squish"
• One-click style effects
• Social media-friendly

Style:

• Slightly less photorealistic than Runway
• More stylized, artistic
• Perfect for social content where perfect realism isn't needed

When to use Pika:

• Social media content (TikTok, Instagram Reels, YouTube Shorts)
• When you need many clips fast
• Stylized videos over photorealism
• Budget-conscious projects
• Experimental/creative effects

6 Example Pika Prompts for Social Content

1. Product Demo (Instagram Reel)

Product spinning on turntable, dynamic lighting, text overlay space, colorful background, energetic, short-form content style

2. Motivation Quote (TikTok)

Sunrise over ocean, waves crashing, camera slowly zooms out, inspirational mood, space for text overlay, peaceful

3. Recipe Clip (YouTube Short)

Overhead shot of ingredients coming together, time-lapse assembly, final dish reveal, bright cooking lighting, appetizing

4. Before/After (Social comparison)

Split screen effect showing transformation, smooth transition, contrasting lighting left to right, viral video style

5. Dance Clip

Person dancing to beat, energetic movement, colorful lighting, camera slightly orbits, party vibe, social media aesthetic

6. Pet Content

Cute dog tilts head, ears perk up, looks at camera, shallow depth of field, warm lighting, viral pet video style

Maintaining Consistency Across Shots

The single biggest challenge in AI video: generating multiple clips that feel like the same scene/character.

Techniques for Consistency

1. Seed locking (when available):

• Some platforms offer seed control
• Same seed = more consistent results
• Limited effectiveness for video (vs. images)

2. Image-to-video workflow:

Generate character image (Midjourney) → Save
Use same character image for ALL video generations
Add different motion prompts
Result: Same character, different actions

3. Style consistency:

• Use identical lighting descriptors
• Same camera lens descriptors (85mm, etc.)
• Same quality/aesthetic tags
• Same color grading references

4. Reference image reuse:

• Generate one great clip
• Extract best frame
• Use that frame as starting point for next clip

5. Deliberate continuity planning:

Clip 1: Character enters from left
Clip 2: Start with character already in frame, continue action
Clip 3: Character exits right

Match action at edit points.

Pro tip: Plan AI video like stop-motion—each clip is a separate take you'll stitch together in editing. Don't expect AI to maintain continuity across generations.

A/B Testing Video Prompts

Video generation is expensive. Test systematically.

Iteration strategy:

Round 1: Core concept

Test basic prompt → does it produce desired subject/scene?
If no: revise scene description
If yes: proceed to Round 2

Round 2: Motion

Test camera movement variations:
- Static vs. moving camera
- Different camera movements (pan vs. dolly vs. orbit)
Keep best motion approach

Round 3: Technical refinement

Test lighting, mood, pacing descriptors
Add cinematic quality words
Adjust speed descriptors (slow, fast, etc.)

Round 4: Platform comparison

Take best prompt, try on different platforms
Runway vs. Kling vs. Pika
Choose platform that nails your aesthetic

Cost management:

• Start with cheapest/fastest platform (Pika)
• Move to higher quality when concept is proven
• Use Runway for final production clips

Using AI Video in a Real Production Pipeline

AI video is rarely the final output—it's an element in your edit.

Professional workflow:

1. Generate clips:

• Use AI for specific shots (establishing, B-roll, effects)
• Not entire videos (yet)

2. Stitch in editing software:

• Premiere Pro, Final Cut Pro, DaVinci Resolve
• Mix AI clips with real footage, stock footage, graphics

3. Add audio:

• AI-generated music (Suno, Udio)
• Sound effects
• Voiceover
• Proper audio mixing

4. Color grade:

• AI clips may have inconsistent color
• Color grade for continuity
• Match AI footage to real footage

5. Transitions and effects:

• Smooth cuts between AI clips
• Add motion graphics, text overlays
• Professional polish

Current best use cases:

• Establishing shots (exteriors, landscapes)
• B-roll (atmospheric shots, inserts)
• Impossible shots (fantasy, sci-fi, abstract)
• Storyboarding and animatics
• Social media content

Not yet reliable for:

• Dialogue scenes (facial sync issues)
• Extended narrative sequences
• Anything requiring precise physical interaction
• Matching existing footage perfectly

Summary: Video Prompting Best Practices

✓ Do:

• Use image-to-video mode for more control
• Specify camera movement explicitly
• Use strong action verbs
• Start simple (short clips, simple motion)
• Describe lighting and mood
• Test on multiple platforms
• Plan for post-production editing

✗ Don't:

• Expect long, perfect sequences (stitch shorter clips)
• Rely on AI for dialogue/lip sync (not there yet)
• Over-complicate first prompt (iterate)
• Forget duration cues (slow, fast, time-lapse)
• Expect consistency across generations without image reuse
• Generate entire videos in one prompt (yet)

Platform quick comparison:

Next steps:

• AI YouTube Video Workflow — Complete video production pipeline
• Runway Gen-3 Review — Full platform breakdown
• Prompt Anatomy Guide — Core prompting principles

Explore:

• 100+ Prompt Templates — Ready-to-use video prompts
• AI Tools Directory — Compare all video AI platforms

AI video is early but evolving fast. Master the camera language, understand platform strengths, and you'll produce shots impossible or prohibitively expensive to film traditionally. The future of video production is hybrid—AI for impossible shots, real cameras for everything else.

📚 Video production and cinematography books translate directly into better AI video prompts — shot type, camera movement, and lighting language all carry over from traditional filmmaking vocabulary.