Artificial Intelligence has given cybercriminals powerful new tools. Learn about the latest AI-driven scams targeting Australian businesses and how to defend against them.
Artificial intelligence has fundamentally changed the economics of cybercrime. Tasks that previously required skilled human effort — writing convincing phishing emails, impersonating voices, generating fake documents — can now be automated at scale and near-zero cost. The result is a step-change increase in the sophistication and volume of attacks targeting Australian businesses.
Here is what the current AI-driven threat landscape looks like, and what to do about it.
AI-Powered Phishing: The Quality Gap Has Closed
For years, one of the most reliable indicators of a phishing email was poor English — grammatical errors, awkward phrasing, and obvious templates. This heuristic is now largely useless.
Large language models generate grammatically perfect, contextually appropriate phishing emails indistinguishable from legitimate communications. Threat actors can:
- Generate thousands of personalised phishing emails from LinkedIn profile data in minutes
- Adapt tone and content to match the apparent sender (a law firm, a bank, a government agency)
- Generate convincing invoice emails that match the formatting of your actual suppliers
- Produce fake but plausible HR communications, IT support requests, or executive messages
The ACSC has documented AI-generated phishing campaigns targeting Australian businesses in 2025-2026 with significantly higher click rates than earlier-generation phishing.
What to do: Do not rely on quality detection. Assume phishing emails will look legitimate. The defence is process (verification procedures for financial actions) and technical controls (anti-phishing email filtering, MFA).
Deepfake Video in Business Email Compromise
Deepfake video — AI-generated video of a real person saying things they never said — has reached the point where real-time video calls can be faked with commodity tools. This has produced documented incidents where:
- A finance team member joined a “video call” with their CEO (deepfake) and CFO (also deepfake) and was instructed to process urgent transfers
- A job candidate in a remote interview was actually a deepfake impersonating the real person
- A business executive received a video message from a supplier contact (deepfake) requesting a change to payment details
The quality of real-time deepfakes in 2026 is sufficient to pass casual inspection — particularly on a compressed video call.
What to do: Establish out-of-band verification procedures for any unusual financial request, regardless of how convincing the video call appears. A code word system — a pre-agreed phrase the real CEO would include in any unusual request — is a low-tech, effective defence against deepfake impersonation.
AI Voice Cloning (Enhanced)
AI voice cloning has been available since 2022, but the quality and accessibility in 2026 is significantly higher. Modern voice cloning:
- Works from as little as a few seconds of sample audio (available from podcasts, videos, voicemails)
- Generates real-time speech (not just pre-recorded playback)
- Is indistinguishable from the real voice for most people in a phone call context
- Is available via web tools for minimal cost
The most common use case remains Business Email Compromise: a fraudulent phone call to authorise a payment, combined with a follow-up spoofed email for confirmation.
What to do: Implement dual authorisation for payments. No single person can authorise a transfer above a defined threshold, regardless of who instructed them. This control stops voice cloning attacks regardless of how convincing the voice is, because the attacker cannot impersonate two different people simultaneously in separate verification channels.
AI-Generated Fake Invoices and Documents
AI tools can now generate convincing fake invoices, contracts, and business documents that match the style and formatting of real documents from your suppliers. These are used in:
- Invoice fraud: A fake invoice from a real supplier (with different payment details) sent to accounts payable
- Contract manipulation: A real contract with AI-modified payment terms, designed to be signed without noticing the changes
- Fake credentials and references: Fake qualifications or references from candidates or contractors
What to do: Implement a process for verifying any change to supplier payment details — always call the supplier using a phone number from your existing records (not from the email requesting the change). Compare final signed contracts against drafts for any changes. For high-value contracts, instruct legal review.
Automated Credential Stuffing at Scale
AI-assisted credential stuffing — using lists of stolen username/password combinations from data breaches to attempt login across thousands of services — now operates at a scale and speed that traditional rate-limiting does not adequately address.
When employee email addresses appear in breach databases (which they will — most Australian businesses have had staff credentials compromised via third-party breaches), automated tools systematically attempt those credentials against your Microsoft 365, VPN, banking, and any other business services.
What to do: MFA prevents credential stuffing from succeeding even when credentials are valid. This is why MFA is non-negotiable, not optional. Complement with dark web monitoring to identify compromised credentials before attackers use them.
AI-Assisted Vulnerability Discovery
Threat actors are using AI tools to accelerate vulnerability discovery — scanning internet-facing infrastructure at scale for known vulnerabilities and generating exploit code faster than human researchers. The time between a vulnerability being disclosed and exploit code being deployed against it has compressed significantly.
What to do: Patch internet-facing systems immediately when critical vulnerabilities are disclosed — do not wait for the monthly patch cycle. The Essential Eight Maturity Level 2 requirement (patch critical internet-facing vulnerabilities within 48 hours) exists precisely because this window is now actively exploited.
Building AI Threat Resilience
The common thread through all AI-enabled attacks is that they are designed to bypass human judgment — they look legitimate, sound real, and create urgency that discourages verification. The defences are:
- Verification procedures that do not depend on a communication channel the attacker controls
- Technical controls (MFA, email filtering, EDR) that operate regardless of whether humans detect the threat
- Security awareness training that specifically addresses AI-enabled threats — not just old-school phishing
- Dual authorisation for all high-risk financial actions
CX IT Services delivers security awareness training covering AI-enabled threats and implements the technical controls that reduce your attack surface. Book a Right Fit Call to discuss your current readiness.
