Product and AI – 4/2024 – Product Innovation: Leveraging Generative AI for Next-Level Experiences
[Music]
just a little bit of introduction I’m
Ross Williams I’m a product manager um
for event tech company um and so today
I’ll be sharing a little bit about my
geni experience and
quite honestly I hope to learn um some
from you ausome folks as well um I’m
located in Austin um you can follow me
on LinkedIn uh but yeah absolutely uh
let’s go ahead and get started so let me
uh let me share my screen here unless
anybody has any questions before uh I
start yeah there we go so yeah so today
we’re going to talk about um let me get
this Zoom set too just one
second okay so yes let’s go and get
started um so what I’m going to talk
about today um is more what you would
think about so we’re going to talk about
geni absolutely um but what we really I
really want to focus on is how do we
think about this as product
managers um there’s a lot of um you know
if you think about this I’m just getting
things set here so I could see um so so
if you think about it there’s a lot of
Buzz around uh geni and there’s probably
uh people in your organization that are
saying why don’t we have it when are we
going to have it uh not even answering
the questions of why would we need it um
which is probably the most important um
and so I’m going to kind of talk you
through that a little bit today um and
then we can come back and and have a
discussion uh I want to be mindful of
your time um you know I think we’ll get
out of here by just before
7:30 um and so I want to be mindful of
your time today I’m kidding I’m kidding
we’ll we’ll probably go about 30 minutes
so um I know people are like oh I’m
leaving right now um and I would be too
but yeah so we’ll go ahead and uh get
started so what is geni let me just kind
of start because I don’t want to just
assume everybody understands this and
and I also don’t want to just be like
let’s just get through this I want to
make sure we learn from this and so
you know when you think about what gen
AI is um it’s really at the highest
level it’s using AR artificial
intelligence to create content ideas you
know including um conversations new
content images video and music and
simply put um it’s generating these
things on your behalf um and so that’s
what gen does you probably have used um
gen it’s some level how many people here
can raise their hand and say they’ve
used gen you can use the little Zoom I
think it has a little hands up thing or
we can use our hands too we’re here um
okay so so yes yeah so um so I’ve been
using it for a while in um and so in my
own you know personal and some work as
well um but it’s using that human int
it’s trying to use basically natural
language processing to make things look
and feel like it was written
you know like a human and it trains
itself over time um and gets better um
and gen is the next step really in
artificial intelligence now if you came
here today and you’re thinking we’re
going to talk about you know what Amazon
is doing with bedrocks and the
foundational models and which one’s the
best or what Aur is doing I’ll tell you
that’s not really what we’re talking
about today they’ll probably be a better
person a more technical person that can
describe these things for you but today
it’s really about as a product manager
how should I think about Ai and how
should I apply my skill set to
it so let’s start with a real question
here um just a poll question to start
with you can use that QR code or just go
to slido.com and enter that code um but
I’m just curious from the group here um
have you introduced gen into any of your
products um so I’m just kind of looking
for a feel there we’ll give everybody
give you you know couple minutes here um
does does like a prototype count like I
yeah
absolutely yeah I meant to actually put
like more like what it means it it could
be in Prototype it could be even in beta
um but you’ve put something in play that
you you know you you anticipate
releasing for is a part of the products
you’re working on
give everybody just another little bit
of time
here uh I’ll tell a dad joke while we’re
waiting so everybody can be
entertained um where do I keep all my
dad
jokes yeah where I I think is where
everybody’s asking right yeah oh
no so I keep them my in my
database so there we
go got past that one oh I got some
Emojis all right we’ll give everybody
just another second here then we’ll
we’ll move
on all right let’s go ahead and move on
and see what we got here
Okay Okay so we’ve got you know it’s
kind of a split um we’ve got some yes
and No No’s a a little bit more dominant
but not not uh not overwhelmingly
dominant um and so yeah this is good um
good to see here we got a nice mix um of
of people that have used it or haven’t
used it yet so awesome okay so let’s
move on any uh any questions so far
all
right so um as a product manager this is
this is product management part so like
as a product manager this is just like
kind of how I perceive things when I’m
working on new products or new
enhancements or things like that I
looked at looked at them through a lens
of what’s called an opportunity
assessment some of these people call it
lean
canvas um some other things um is as
well but this is what I use as kind of
like my first step I actually did one
for the product um that I’m talking
about but I’m not going to include all
that that details here um since I think
this is being recorded so um this
basically lets me as a product manager
look at anything not just gen AI um but
look at anything and say what’s the
opportunity um and this is actually
meant for um in some some circles where
you might have like a product counsil or
or some product group that helps kind of
lead the road map for your products or
um kind of you know gives you that
approval that you might be looking for
from upper management um in some cases
and it’s just a very high level you know
what are the targets we’re looking for
what are the customers are they new
customers are they current users um
what’s the problem statement that’s like
to me the most important thing is like
are we solving an actual problem um
maybe a bulleted list of like high level
features like what are these features it
will include um it also includes
competitors so what are competitors
doing are they actually doing this
already or not um and that holds true in
gen as well um and then you know Revenue
stream which is where you know you’re
going to get the Buy in from is is this
going to make this money is this table
States what is this doing for us like
where does this put us in terms of our
re Vue or is it just a you know thing
that you have to look at because that’s
how your competitors are beating you um
you know when you analyze like when loss
data or other things like that what are
the cost factors yeah we’re going to
make money but how much money we have to
spend to make that money um and then
what are our risks you know like if we
do these things what are our risks and
then I usually just have a product
recommendation which basically just you
know what the product team recommends
because as product managers we should
always have an opinion um and so we want
to make sure that we’re um you know
making the right opinion and that could
be a no could be a yes um but this is
more like a step back it’s not about AI
this is just kind of like a plus thing
to think about um is you think about
your your opportunities in your own
product worlds um what this might look
like any any questions um any questions
on
this okay and I do use this with usually
uh just so everybody knows this gets
used as like kind of a first cut with
the executive leadership team or maybe a
team below that to make sure that we
have some Buy in or understanding of it
so you want to make sure that you know
you’re showing this to some people
before they um before you know you show
it to a larger group right you want to
build that
stakeholder uh alignment a different
topic for a different day but um sorry
yeah do you find that any one of those
particular boxes um gets more play in
the discussions when you’re presenting
it to
leadership yeah the revenue
stream
okay yeah that’s where like once I show
it by minutes two minutes later
somebody’s hand goes up and says let’s
focus on that Revenue stream or the cost
Factor
um and again this is not like a full
like document of outlining the financial
models and all that stuff this is more
like hey do we want to go investigate
this further um so it helps me to manage
my time as well so I have five of these
things I want to be able to have I want
the opportunity to be to our leadership
team um and then we may pick two of
those or three of those or maybe back
burner some um but yeah um and then on
the technical side to your point it’s
usually the risks so development teams
will usually want to look at the risks
um involved with that if they’re mostly
sometimes technical
risk
um oh I see we got somebody okay uh any
other
questions all right so let’s keep going
now okay so when it comes to the geni
part you know you want to really focus
on understanding those user needs what
are the user needs what why would they
want to use gen what are those Discovery
you know as you’re doing discovery on
the new feature like you want to
understand the use cases um and the
problem that you’re solving for um does
gen solve it better than current
Solutions um is there other ways we
could solve this problem that may be
less
complicated um or could be used it’s
cheaper faster Etc um you know you want
to always try to look at those
Alternatives as product manager and not
just go in with this is the end all be
all right um you want to think about the
Alternatives and then I the last one I
think is the most important of this one
is you think about your user prop and
and value prop is ensure the value
proposition is clear compelling and
addresses a real Market user problem you
don’t want to go in and get caught up in
the hype cycle that is Gen um and then
be like well how are we going to use it
and you’re like I don’t know we just
need yet right um You Want To Make sure
you do it correctly um because you can
do it incorrectly as well so any any
questions on
this all
right so this is what we’ve kind of
talked about a little bit is one of
those is that business uh performance
and revenue like how are we going to
make money um what is the you know we
want to think about our outcomes um over
output what is it that we think we can
make money off of or does this just help
a client experience better um in my
world uh we have a lot of people that
are content creators um in marketing and
so this is like a really interesting way
for them to create content within a
product that they could use as a I I
like to call it as fire starter right I
don’t think geni is going to give you
your complete answer I think you’re
going to take it and full elements of
that from the um the Gen responses um
but you’re going to use it to maybe
articulate better and form a better um
you know question or I’m sorry a better
answer to a question um I think it’s
great for Content management when you
think about like events like hundreds of
sessions and descriptions of all these
sessions um you know you want to be
clear about this is what we’re going to
do
um and this is like phase one or this
could be a different phase but you want
to think about phases as well um in
terms of like the business performance
and the
revenue this is a you know this is one
of the oh somebody has a
question hi Ross yeah a quick question
when you go back to the last slide about
Revenue so the question is is either a
product that you’re enhancing or you’re
actually creating so when you have
something new you have additional cost
your first year second year piece so
everyone I’ve noticed sort of kind of
put in the billion Doll Market meaning
share piece of it but okay that’s what
exists what percentage do you usually
take within your first year second year
phase because some have some like
organizations like yours already have
resources and some won’t have those
developers sitting there ready to you
know get to work so right right so I
think from a so let me make sure I
understand the question you’re you’re
kind of saying like if you’re going from
a zero to
one right you Z to one with a new
product start from scratch development
versus enhancements my question is I
really like to know with the business
performance and revenue that little
bucket is what percentage do you use
because when you use market value for a
billion Doll Market let’s say for soap
soap
dispensers right you know something like
that then what percentage do you take do
you take 10% off do you take 15% to show
your Revenue
possibility yeah so that’s a good
question so in my case I can only use my
example um it was not a zero to one it
was a product that was already
established and in place um I haven’t
done a zero to one yet um with geni just
hasn’t been within my my Pro my work um
but when you when I do it for like
something it’s already established
things are much easier right because you
can then look at your user base and say
okay if we bring this in um you know
what will be our attach rates right um
how many clients will pick it up
especially if I charge for it right if I
just throw it in then it’s just
something we can go brag to the market
about but um but if I’m trying to build
in Revenue you know I when I’ve done
other Financial modeling um I usually
start the first first year at about a
10% attach rate maybe 15
tops um and then you know and then I
kind of build it from there um these
could be answered different questions
Maybe by different markets as well um
but I you know I don’t want to say I’m G
to have a 100% tach rate in year one I
don’t think that’s feasible I also think
um so I don’t know if that aners part of
your question but let me answer the
other part about the zero to one
unless the Gen AI is absolutely critical
it may not be a part of that mvp you may
have other things that you have to build
out first like the foundation and the
structure and the back end right and
then you can bring that on um is you get
more funding or your user base grows um
that’s a that’s a kind of a product
question B based off a lot of different
variables I would say um but I think
that’s how I would look at it is like
does this need to be there from day one
or can we hold on this while we just get
out the product to
Market so I don’t I don’t know if I’ve
got a good answer for your question
honestly Heather but I’m I’m trying I
think it is I think we use pretty much
similar numbers as well so thank you for
answering that yep
absolutely um so so back to the
technical side um you know you got to
have those technical requirements in
place and that’s going to be your
development team kind of like we just
talked about right do they even have gen
experience does our current architecture
account for it do we have an API that
can talk out um you know what additional
resources are we going to have to get
some more you know um people that are
more closer to AI products to think
about what I just was referencing
earlier around like okay we want to do
this but what platform you can’t talk
what platform are we on but more
importantly can that platform
communicate with us internally to pull
in that
information um you know what are the Ben
what what are we going to have to do
there you know to help with that because
then it’s going to be more like okay is
this a six you know week thing or a six
Monon thing um okay I see I got a couple
of hands raised here uh let’s go with
Joe I think you were
first hey just uh FY on the resources I
uh play with these apis on my free time
I you know I don’t know been doing full
stack like 10 years now uh it’s pretty
standard pretty standard rest apis most
season full Stacks can consume it
pretty like without a lot of difficulty
just I use the anthropic one I use the
open AI one and I use like a local one
called mistro anyway that’s it no and I
appreciate that um I think where my
thinking was on that is like do we even
have an API that can communicate into
that system right is it a closed
architecture where we’re only sitting
within our own you know AWS Cloud do we
have apis what do we need to do there
that just that technical you know you
want to answer those questions as soon
as possible to say yeah this is doable
or not because you don’t want to keep
going down the path if it becomes a
thing um where you know you have to end
up um you know kind of stepping taking a
step back because you think through some
of these feasibility and integration
issues up
front I think there was one other
question or somebody had their handris
Joel’s got his hand up Joel
oh let me you’re un you are uh you’re
unmuted or you’re muted I think yep I
was just waiting yeah yeah go ahead yeah
so quick question really back to the
business justification I saw an
interesting Tech talk uh Ted Talk the
other day um let me see if I could find
who it was got Nam Mustafa sullan
talking about what’s an AI anyway and
where this is headed and you know
something he mentioned is I’ve been
thinking about this is it seems like
these large language models effectively
could displace the entire user interface
of everything we do in the business you
know where instead of having to have
menus and structures and everything you
just ask the system to do the thing you
want it to do um I’m curious how you
would think about the cost justification
of something like
that you know where instead of having
the whole ux through user experience and
Graphics thing you just have a system
that says oh you just want to talk to us
fine tell us what you
want yeah wow I honestly that is I
haven’t heard that um maybe that Ted
Talk I need to look like although if you
do have a good Tik Tok to share I like
those as well oh sure I’ll drop it in
the chat if you want um but um but it’s
inter yeah that’s very interesting um
you know I’m seeing by the way as an
example just I’m seeing the beginnings
of that for example um atasan put into
uh jira they basically displaced their
normal well not displaced but they added
into their text query engine that
instead of having to know how to build a
query you just talk to it llm style to
say hey I want to know about this and
this and this and it constructs in the
background what it
needs that would be very helpful for me
um okay yeah so um because that’s kind
of hell sometimes um but I I mean yeah
that’s a good example of that I mean I
think we will see more like where you
like kind of use that chat query right
like show me XYZ or and that maybe it’s
just a prompt there the user experience
on that I think would be something that
you would probably have to think about
as a very slow roll out um with maybe
only your very Savvy depends on how
Savvy your client base is
too lots of people even when we change
you know I’ve been a card of like uh
refreshed themes you know we’re
refreshing the UI or the ux and the
buttons change at all but the colors
do what are you doing I’m so used to
that yeah we just changed the color to
make it better because we needed to to
you know for Ada or whatever um and so I
I think it’s interesting but I I I don’t
know that I see that happening except
for maybe more technical Savvy products
like what you have with
alassan other that um you know software
platforms or services that are doing
important things like compliance and uh
other areas I think that’s going to be
my prediction would be that would be
that’s going to take a long time to get
there I guess okay I don’t want to I
don’t want to hit the rabbit hole too
deeply on this one but but imagine Now
with an application some web app that
has you know a few dozen main pages and
a whole bunch of functions in it that
people have to remember what’s where to
do things y yep getting people to take
the first step might be a bit but the
minute they realize they could say hey I
want to get a report like this and it
appears without having to know where it
is and the part that just to bring it
back to the product design
thing it almost seems like like that sub
subverts a lot of product
identity because if a bunch of people
start doing that then it’s all just
about asking questions there’s no face
if you will that they’re interacting
with in terms of a product thing so I
don’t know I just figured I’d toss that
out there for thinking about I think
it’s a very interesting topic um yeah so
thanks we still have four hours left so
we can you know we can go on for a long
time
here I’m out
let me keep going the one of the things
that like
um uh something okay um one of the
things that is going to come up right
off the bat from your legal teams and
maybe even your other teams is what
about our data data and privacy
considerations where’s the data going to
be stored how long is it going to be
stored is it going to be
anonymized um all these things are very
like and you have to do it on a caseby
casee basis um you know in terms of how
you’re going to input it but if you
think about it holistically if you’re
going to use a service like Amazon web
services and bedrocks or any of these
other ones they’re going to have data
and privacy concerns already addressed
or they should have um the one we picked
was with Amazon and they had a whole
thing on you know
security privacy all that stuff you
still have to know it
um but this is going to be very very
important um because clients are going
to be putting in data that could be you
know something close to their
organization or company so you want to
be careful about that and you’ll have to
absolutely have a um a team um or a
person that can help understand that for
you and make sure you get that sign off
before you would even move into you know
obviously into production but what
that’s a big deal for something to
really think about and consider um as
you move
forward um okay
so then we talk about user
experience um so then you got to think
about okay we need this everybody’s in
love everybody you know they’re crying
uh sorry real quick are you sharing your
screen oh yes I thought I was okay yep I
think I see it now
thanks okay can you guys see
it okay yes I don’t know what happened
there was a zoom popup um and I thought
I was already sharing but it as long as
we’re good um I’ll keep
going um how does it fit into the
journey you don’t want it to look like
it’s a hge Podge thing that you just
added right you wanted to make sure it
looks good within your format you know
if you’re using text for Content
creation where’s it going to fit within
your UI as you use it you don’t want to
be overly complex kind of back to the
whole thing right you want to be very
simple and easy to use um and then you
also got to think about intuitive
interactions and I think about here is I
think about like do we want to have
prepr preset prompts that help answer a
the 8020 rule right or do we want to
have it where it’s
open-ended um or a combination of both
um you want to think about how the
interaction is going to take place
within the geni within the product um
and how that’s going to be impacted and
and useful by users and obviously you
got to do some level enablement and
training along these lines as well but
you want to think about how that’s going
to be how that’s going to impact that
client experience and that user
Journey um this is probably a bigger
topic for a different discussion um but
you got to think about the bias you know
and the ethics to this um what are the
ethical implications of using geni what
is that going to do for you um or you
know could that be something that comes
back on your company youve we’ve heard
about this right Google or Bard or
whoever and they find out that it’s
being ex skewed somehow um you know you
want to think about these types of
topics um as well and you want to be
proactive in identifying them these are
probably conversations and and that
you’re going to want to have with um you
know whoever supplying you this the
models the foundation models um because
they’re going to be obviously already in
the leads they and they should be on
this um but you just want to have this
as something as a discussion point when
you’re having those conversations with
those third parties about who you want
to generate um who’s going to generate
that for you and how do you help um
fight um these types of things that
could be come up in your product as well
feedback loops so feedback loes are
going to be critical um you know if you
use a product like um pendo or any of
these like feedback products within your
organization um you’re going to want to
get feedback from clients on how they’re
using this um how they’re delivering you
know what what is it delivering is it
delivering do you like it um you know I
don’t really think this is like needs an
NPS score a net promoter score I think
it’s more like you want that individual
you know is this making your job easier
right is this something I can do um to
make my life easier um but this feedback
is going to be crucial for as you
iterate and improve it over
time so um you know we kind of had a
long discussion here and I think this is
my yeah that was my last slide was kind
of going through that so you know
hopefully what I’ve done here is just
given you and I can share these slides
out too but my goal is is a product
manager to think about what are the
things I need to consider um when I
think about
geni and you know kind of coming back to
the whole conversation around does it
solve user problems um you know you want
to have those client conversations I
think that’s the most critical um to get
it right up front um but then as you get
past those Sation and you make those
decisions to say hey guess what gen is
something we should do I I these are the
topics that you should look at um as a
product manager to make sure that you’re
taking care of these um so you can have
a successful you know integration and
take that product to that next level um
when you want to generate um use
generative AI as a part of that a part
of your product so yeah I got us we’re
we’re about 340 so it’s not too bad
um but I don’t know that’s uh that’s
that’s my topic but is there any
questions um anybody has that they like
to uh to ask I see a lot of chats but
let me let me see if maybe I
can get those I don’t know if there’s
any uh for some reason I can’t on the
chat it’s mostly people just sharing
their LinkedIn profiles so okay okay I
do see that we got a couple hands raised
I think Joe in was first okay go ahead
Joe so Joel brought up the topic of uh
having a user interface where you could
just chat and ask for things um
currently the state of generative AI or
just as an example I’m sure there’s
other uh it it does a lot of
hallucinations it’s not what we call in
the computer science world deterministic
meaning I have these series of clicks
and I always get this output um
that can be hugely frustrating to a user
uh I think we’re a long way from that
glorious Oracle where you just speak to
it and intuitively knows with your
language and the context and your
background exactly what it is you want
um and then with respect to the
uh to the uh uh bias situation um a lot
of times these models it’s it’s garbage
in garbage out right like if you train
it on like you know some really nasty
stuff you get nasty stuff out so um just
be mindful your training sets and what
the data is getting trained on um and
you know once you create an AI it’s a
like a Persona so that Persona has to be
on brand with your company um at least
those are those are my thoughts on all
that
jazz okay yeah those are awesome um
awesome inputs
thanks um any other questions
well I guess I had my hand up so I
should respond since I was named
uh uh yeah I think that I agree the
hallucination thing is a problem the
industry is actually working on that and
there are some beliefs that it’s going
to get dramatically better in the next
half year or year um but even past that
what I was talking about was a very
limited set of information right that if
if you have um basically a prompt or an
information set that is here are the
things are application knows how to
do and that the answer isn’t necessarily
provided right to the user the user give
it just as you would for like a Google
search some you type in or you say hey I
want to do this and the llm delivers
back to the application that’s this
function that then gets called as if it
were basically it’s an interactive menu
system that I think the hallucination
thing is a lot less of a problem because
then if it happens to pick something out
of th thin air and hallucinate it it
just looks like a question it can’t
answer as opposed to handing back bad
data in other words it can’t we can’t do
something so on a negative side yeah it
could occasionally say well I can’t do
that when the system clearly can um but
I suspect the risk is much smaller
because the input data is so small and
it’s a contained domain um the question
that I uh would like to ask actually uh
for uh uh Ross
is people have maybe a passing
understanding of this stuff at the this
point generally and and product folks
are so busy with you know I’m dealing
with current features and current
product whereas this looks like it’s way
out there even though it’s hitting us
like a ra Railway train any thoughts
on ways to help product folks and
product
organizations get the right kind of
backdrop with generative AI so that they
can make Intelligent Decisions about
what to do with it from product design
perspective yeah um there is I think um
when you look at some of the products
specific I’m going to call it well
there’s like product school there’s
pragmatic there’s some of these
organizations um pendo actually has a
training course you can take um on
generative AI I think it’s free or it
used to be free um I took it a while
back um but there are a lots of courses
that are out there you know specifically
for product managers to get
understanding because you again you
don’t need to know you know the
technical aspects of it maybe you’re a
technical product manager and you but
you know you need to be able to speak to
it fluently and understand what it
means because it’s probably going it’s I
would say that whatever industry you’re
in um verticals whatever um gen I is
already being impacted or it will be
impacted within the next one to two
years um and so you don’t want to be the
ones that are left behind um you know
especially if you’re like on some of the
Gardner magic quadrants or forcer wave
reports or any of that stuff you don’t
want to be the one that’s called out for
not having these things and I’m not
saying that to say we have to have it
because we have to have it we need to
have it because it solves this Market
problem it’s solves this Custer problem
so yes everybody I think there’s a big
hype cycle around geni I do think like
it’s overplayed to some degree um but I
think it’s more about as a product
manager we have to think about how does
this impact our business um does it help
hurt hinder how can we use it in our
products and then what problems are we
solving for so I would say that at a
high level that’s kind of where I see it
um from you know that higher level view
thanks and one other thing if I could do
a little bit of a pitch because I have
sort of a bunch of tech background as
well is that
um helping people to understand that
it’s not just a chat engine I think is
really important sort of this example of
as a user interface or as a way of
making things happen that it is
essentially this thing you can integrate
into your product as a part of the
product and not just a flashy Chat thing
that you put on top of it um and because
I think that’s to me that’s where the
really significant stuff is going to
happen you know past like the three to
six month Horizon where everybody’s
putting chat Bots up so right thanks
yeah the chatbots is like from a product
I mean that’s kind of the biggest thing
you know right now that clent they think
they can replace their customer support
team which they can actually I know
everybodys right until you get hit with
yet another lawsuit for selling a car by
for a Dollar by your chatbot you know
yeah case thanks appreciate it yeah
exactly yeah but I think that there you
know there will be more use cases um but
like I said I would look at product
school I at pendo I look at I know
pragmatic has a has a now not all these
are free but like some of them are free
but like if you’re looking from a
product perspective I know pendo offers
a training um that I took um and then I
think pragmatic does but I think it
costs money but um but yeah that’s you
know there’s
lots I would focus on the products stuff
yeah I would from the product lens
product management lens right so okay
hey folks Heather has been has had a
hand up for a while oh I’m sorry uh
Heather is that who you
said yeah thanks um yeah I just wanted
to bring this back into the product
World here for product people managers
um and I know Ross you said you wanted
to learn from others as well so throwing
it to the entire group how are you guys
using it um for your day-to-day to make
life easier have you used it to make
anything in your workflow EAS yeah so
I’ll answer but I want to hear other
people’s answers too um but for me it’s
been mostly um because I work with a lot
of event
people um they use it for Content
creation of like they have a let’s just
say for example uh they have a session
on you know bank fraud in America I’m
making something up uh but they would
put in you know write me a description
for bank fraud in America and then it
would give them back some resp responses
again it’s not to be the end all be all
that they would just copy and paste in
to that it would be more like okay this
is some good things that we can talk
about like it’s kind of like I keep
calling it a fire starter because I
really think that’s what geni is is
really good for is starting those
conversations but you still need that
human intelligence to figure out what is
the right things to say and how do we
want to manage that expectation um but
for me it’s been content creation but
I’d love to hear you know others talk
about maybe how they’ve used it as well
I can tell you on mines I’ve used it for
PBI refinements um it’s been good for
PBI refinements as well and then if I
just use it as a check for if I’m
missing anything sometimes you know
feature wise that but you do have to
be very well written I would also say
your input also as well so sometimes it
misses it if you miss
it right it’s the whole garbage in
garbage out thing I think Joe said
earlier
cool we did have we did have a couple
comments in the chat that I think uh
I’ll read out before you hear Ross the
first one if you want to resp just have
some thoughts to respond to it so Rachel
Ry wrote I think it’s helpful to view
geni as a tool to retrieve or query
unstructured data it seems most useful
at this point being used internally with
the large Corpus of unstructed data to
provide insights into Data that has been
largely dark
data yeah that’s that’s a that’s a
really good point too um I haven’t
delved into that part in looking at you
know using that in my current career um
where I’m at but um that sounds like a
very good use case for it um I don’t
know if anybody else has any experiences
with
that yeah but and then the second
comment Ben may also wrote that the hype
train is so real our execs tossed us a
product based only on hype quote we want
to be the first in our business to have
it exclamation point unquote then we had
to go figure out how it would actually
be useful to our
customers wow um I don’t know what to
say that Ben I mean that’s the cart
before the horse right um and so I think
as product managers part of our role is
to say okay we can let’s talk let’s have
let’s talk about this I’ve had this in
other things too uh I’ll tell you in
another life we had audit trails and
there was this idea bubbling that we
could use
blockchain um to do some of the trails
because of the way the audit The
blockchain Works uh so I had kind of the
same thing on that hype cycle a little
bit um ultimately we didn’t do it
because I was able to get ahead of it um
but yeah I mean it’s happening because
that’s all CEOs hear about right now is
Gen and there you know especially if
their private Equity they’re talking to
their firm you know and they’re like
when are you guys doing this
so uh my heart uh goes out to you uh but
ultimately you gotta kind of feed that
back into the group to say you know okay
what problem are we going to solv with
this because if the problem is well
that’s your job to figure out that’s
could be a good answer to say well let
me go figure it out let me go see if we
actually solve a problem here um and
then you could come back with that but
yeah I I I’m not shocked I’m sure this
is happening in organizations all across
right now about how are we going to deal
with Gen and you know and more
importantly when are we going to have it
in the
product so Ross um I think what was
interesting and you flashed a slide
earlier that was about um the
opportunity ass assments I mean right
there right I would assume if the
opportunity assessments are consistently
being
followed um you know some of those
things would automatically drop off to
say hey it doesn’t fit in our
Matrix right because there’s really
nothing that we’re adding as a value to
a customers even though and maybe
there’s a there’s a value in terms of
like you know just the hype or the
marketing side of values maybe there’s
some value there but net net will
probably fall through anyways because
the ROI and all that might not be good
enough exactly yeah I mean it’s not it’s
not
free uh the Integrations are I mean you
know depending on where you’re at
technically too in the company right do
you already have an API that can talk do
you have these things in place and
that’s why I come back to this
opportunity assessment like you said
because this is what gets generated by
the product manager and you know this is
something you could even use to say here
you know look here’s what we’ve done
this is the problem statement these are
the features um but here’s our cost and
our risk and our Revenue stream and see
if that matches what you know the
leadership team is going to want to you
know push
forward so I mean I guess just as a
quick followup um I’m wondering if most
folks uh in your companies or
organizations have a product C
it’s typically that’s the role of the
council to kind of look at all of these
in a consistent objective way I use some
scoring
models yeah so for me I it’s an absolute
necessity so we don’t have these types
of
conversations um with what what is it
the highest paid opinion or whatever I
can’t remember what it is but that gets
you away from those now is that still
going to happen absolutely but I think
you need a process around it which is a
whole different topic um right but if
it’s a small company it might just be
hey get in here I’m the CEO let’s figure
this how we how you going to do
this there’s another
topic stakeholder engagement and all
that
stuff so we had another
uh another comment from Joel about
hallucinations from the chat I’ve
already seen some pretty interesting
hallucinations from asking llms to
analyze data and Joel by the way I’ll
second that so some paths for verifying
results is in my opinion required until
hallucination problem has some trustable
solution so yeah I guess Ross any
thoughts on quality um checking results
reproducibility man that’s a that’s a
tough one um that’s probably a more
technical question um because of how
those questions are being asked into the
the the
prompts um but you
know I
think depending on the use
cases that could or could not be a
problem um if you have very specific use
cases where it’s going to be used maybe
that’s less of a problem I don’t know
um but I could see it you know if you
have specific Parts if you’re just going
to open it up to the whole product I
could see where maybe that happens but
if you have like specific use cases
within the product maybe that becomes
less but that’s just kind of my
assumption but I don’t that’s not based
off of any facts yeah if I could I’ve
I’ve seen cases where literally you hand
it a table of data and ask for a sum of
a column and it just makes something up
and hands you a number yeah so I mean
that’s not a sophisticated use case it’s
like here’s a column of numbers what’s
the
sum and sometimes it’s spot on so you
know it’s that’s my concern it’s just
that you can’t know when it’s going to
hand you nonsense and when it isn’t so
to I think it’s great at pointing out
possibilities right but I would always
want to verify if it throws out
something positive or negative and
you’re going to take a decision based on
it I currently if it’s worth any amount
of money I’d probably cross check it
these days yeah maybe
paranoid go ahead sounds like the
product manager just sums up a number
and whatever number I need it to be is
the number I use right it’s the perfect
Dilbert machine
right
exactly okay uh well we’re almost at the
time which I’m I’m I’m really excited we
only I thought this would go a little
bit shorter so I’m I’m happy that people
had questions but if there’s any other
questions um you got a couple more
minutes you can ask go also uh you know
hit me up on LinkedIn I’m happy to
answer any questions um I think also you
know there’ll be topics where I think
the community you know this group here
will be looking for more for more people
uh much smarter than me to do some of
these as well so um you know that’s
another another thing to think about too
thank you Ross uh completely appreciate
this was great um was a great start to
kind of the discussion get the thoughts
flowing around this uh and a structure
around how to kind of think of GM
products uh thank you once again for
signing up to be the first one uh we do
have a few more people that we’ve
reached out to uh but absolutely would
love to hear from the group that’s
already here as well um you know so that
we have a backlog and that way every
month we’re not searching for new topics
uh obviously between uh myself and and
Matt we’ll kind of uh try to arrange uh
the topics so that they’re interesting
uh every time but please uh if you
interested once again in presenting if
you have an idea or a topic you might
want to talk about uh please feel free
to uh submit that either via chat or
send us an email um the other thing we
had talked about uh was at some point
seeing if we can kind of just do like a
quick you know um I guess kind of like a
Q&A but more like a here’s my problem
how how would I use AI to do this so
just kind of like as a community helping
each other out um that hey you know this
is my product here’s an idea or does
someone have an idea of how I can
incorporate um some AI into my product
area so again it can be pretty
open-ended um kind of like a you know
fast um this thing but anyways so we’re
going to try that out as well uh maybe
we’ll try it out like a 15 it uh in one
of our meetings uh upcoming
meetings um but that we kind of
hopefully I don’t know how that resonate
that idea uh with everyone um but if you
feel like that would be something we
should try uh please let us know um
maybe we’ll start with a 15 minute
thing um Al righty
um thank you all once again uh we will
look forward to by the way apologies
from on my side I was having some delay
in joining today um and thanks Ben for
covering um we will uh continue the
conversation a month from now but please
feel free to use um the access to slack
that we’ve now received uh and obviously
um I think that’s a great Forum um
so thank you so much okay a thank you
have a great day thanks everyone
By harnessing Generative AI, businesses can unlock unprecedented opportunities for creating next-level experiences. In this discussion, Ross will cover why he and his team decided to implement this and how they did it for a better client experience within an eventtech product.
Speaker: Ross Weems
During his 12-year tenure as a Senior Product Manager, Ross Weems has been continuously leading initiatives that combine profound technological insights with strategic business applications, especially in the B2B SaaS arena. At the heart of his professional philosophy lies a dedication to integrating complex systems to enhance user experiences and optimize operational efficiencies. Ross enjoys flying drones to capture awesome pictures whenever he has the opportunity.