AI Consulting

This is a primer for delivering value in AI Operations.

Suggested Skills

  • Extensive prompt engineering with Bing, Bard and GPT4 (including GPT plugins, sandbox and shots), prompt templating and chaining
  • Microsoft Copilot will be significant with tight O365 integration, but is still in private beta with only 1k users worldwide
  • LLaMA familiarity
  • Midjourney/Stable Diffusion familiarity
  • RPA (Robotic Process Automation) experience is a plus, eg: Pega
  • Zapier or Make (integrations)
  • No-code app experience (eg. Bubble, Retool, Notion, Airtable)
  • Github Copilot (coding)
  • Basic coding skills (Python, JavaScript) to understand and troubleshoot AI generated code
  • Reengineering/first principles thinking mindset

Services

Up front services

  • Chatbots
  • Automations
  • Custom data to train models
  • Prompt libraries

Retainer services

  • Monitoring & optimising costs
  • Updating models & techniques
  • Regular tests to track performance and address prompt drift
  • Prompt engineering training
  • Prompt engineering 1-on-1 assistance: to individual users for specific use cases.

Pricing is determined by

  • Consultant expertise
  • Customer company size/potential benefit

Finding AI Use Cases

Initially, don’t look for ideas. Look for problems/frustrations that:

  • an intern could be trained to perform satisfactorily
  • are performed regularly and consistently enough to warrant automation
  • are text-heavy

Eg: categorise and draft email responses for review.

Prompts

  • break down into sub-tasks where possible (chaining)
  • diverse data and examples give better extrapolation
  • keep a human in the loop (review every result or at least sample results)
  • test for hallucination
  • test for bias
  • keep a log of prompts, settings and results
  • monitor for prompt drift

Cloud vs On-prem

Cloud risks include:

  • IP risk: 
  • Regurgitation risk
  • Information risk
  • Inference risk

Sample Hardware Options

Model Size

GPU

CPU

RAM

TFLOPS

Model Storage

Total Storage

$AUD

7B

RTX 3060

Ryzen 5 5600X

32GB

12

4GB

4TB

$2,500

13B

RTX 3080

Ryzen 7 5800X

64GB

17

7GB

4TB

$3,500

30B

RTX 3090

Ryzen 9 5900X

128GB

36

16GB

4TB

$5,500

65B

T4 16GB

Xeon Silver 4214

128GB

100

32GB

4TB

$10,000

65B

V100 16GB

Xeon Silver 4214

128GB

150

32GB

4TB

$15,000

65B

V100 32GB

Xeon Platinum 8280

256GB

300

32GB

4TB

$25,000

65B

A100 40GB

Epyc 7742

4096GB

408

32GB

4TB

$35,000

65B

A100 80GB

Epyc 7742

4096GB

717

32GB

4TB

$50,000

65B

A100 80GB

Epyc 7742

4096GB

740

32GB

4TB

$35,000 (tinybox)

Comments

(Will be hidden)

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.
CAPTCHA
8 + 6 =
Solve this simple math problem and enter the result. E.g. for 1+3, enter 4.
Are you human?