GenAI: still failing to summarise information
According to the findings of a trial for the Australian Securities and Investment Commission (ASIC), generative AI is not good at summarising information compared to humans.
The trial, conducted by Amazon Web Services, used Meta’s Llama2-70B, which isn't the latest model, but with up to 70 billion parameters, it is a capable one.
Llama2 was instructed to summarise documents submitted to a parliamentary inquiry. It was explicitly instructed to focus on what was related to ASIC, such as where the organisation was mentioned, and to include references and page numbers. Alongside the AI, human employees at ASIC were asked to write their own summaries.
AI-generated summaries scored 47% on average, significantly lower than the 81% scored by human summaries.
AI struggled with context and relevance, producing "waffly" summaries.
What does this mean to marketing?
The bottom line is that you cannot rely on AI to summarise documents acutely. I have covered this subject before in an article that highlighted AI's tendency to skip data in the middle section of a document when summarising. To help overcome this, you can break the document into smaller parts and ask the agent to read it twice. But ultimately, human oversight is still required.
Mircosoft announce next phase of Copilot
Microsoft has announced its next phase of Copilot, and you can find out what’s in store with CEO Satya Nadella via LinkedIn, 8 AM Pacific Time on Sept 16th.
What does this mean for marketing?
Microsoft Copilot offers significant benefits to marketing departments, including content creation and optimisation, SEO, brand sentiment, data analysis, audience segmentation, and campaign and workflow optimisation. If your marketing department uses Copilot, this is your chance to be first in the queue to determine how the next phase can benefit you.
I will be attending the event and will report any significant changes.
Google Photos to roll out Ask Photos
Ask Photos is a conversational search that aims to understand the context of your photo gallery and highlight relevant details to help you find specific memories and uncover information about your life.
Early access to Ask Photos will begin with select US users on Android and iOS.
What does this mean to marketing
We are in an era where search doesn’t require metadata to find what you’re looking for. This means anyone with vast amounts of (badly labelled or not labelled at all) imagery can start to organise and make good use of it. Need a photo of the CEO presenting in Barcelona in 2013?
Salesforce building autonomous AI agents
Salesforce is taking a step forward with Agentforce, its new platform for building autonomous AI agents. Salesforce promises that custom agents will handle basic tasks and queries autonomously, from booking meetings to upgrading chatbots into full conversational partners.
Agentforce will allow Salesforce customers to quickly set up and deploy autonomous AI agents designed to deliver autonomous sales tasks.
What does this mean for marketing?
For Salesforce customers, now is the time to understand how to leverage Agentforce as it becomes available. Being early to the party could give you a competitive edge.
300% price hike for Canva customers
The graphic design platform Canva has announced substantial price hikes for its Teams plans, with increases reaching up to 300% in certain markets. Canva has recently added a number of generative AI functions, and it claims this has added to the platform's value.
What does this mean to marketing?
Don’t panic... yet. The new pricing isn't expected until December.
And finally…
The US, EU, and UK signed the world’s first legally binding international AI treaty, emphasising human rights and accountability in AI regulation.
You’re up to date with this week's AI Marketing News :) Feel free to share the news with your team.
David Sloly
Subscribe to the AI Marketing Newsletter
Comments