How to use Webhooks and Zapier (or another API tool) to improve your reply rate

  • 16 September 2022
  • 8 replies
  • 176 views

Userlevel 5
Badge +4

I thought I’d “walk-the-talk” and share something I’m really proud to have unlocked today...

The Challenge: 


We have really great reply rates on our community. We typically see >75% of questions receive a reply within a give month. Usually it’s more like >80% and we strive for >90%. And our median time to reply/answer usually ranges from 1-4 hours. These are key success metrics for us, as community sits within Customer Success, but also just common sense of “positive experiences (from making people feel seen/heard) will drive adoption and engagement of our community.” 

Answers are a lagging indicator to replies. We see that an increase in the % of questions receiving replies correlates closely with an increase in the % of questions marked solved.

But our success has presented a new challenge. As adoption and engagement of the community increase, our question volume is spiking. We’ve seen >10% growth in total questions QoQ and last quarter it was >20%. This is making it hard for us to keep up: the number of replies and answers is the same, but that means our % of questions receiving a reply has dropped down (75%) to well below where we want to be (>90%). 

The other thing to know is that our external community members are driving the majority of this success. Each month, non-staff users account for well over 80% of our replies and answers. And it’s often >90%. Our community is helping us scale customer success through a robust knowledge base and active Q&A forum that deflects support tickets. (Oh yeah, I’m the only person on the community team and we have no other moderators, just staff using community the same as our customers and partners). 

But again, success has presented a new challenge. A lot of our contributions are the result of a super user program and gamification. However, we’ve seen engagement fall off after someone reaches the super user status. When we researched it, we learned that it’s because the community helped them in their career! Contributing on the community sharpened their skills, cultivated expertise, and earned them recognition on the community, leading to promotions and new jobs. 

TL;DR: This quarter, we’ve targeted scalable ways to keep up with our engagement growth. How can we extend our gamification to keep super users motivated and to incentivize more users? 

 

Solutions:


We’ve been extending the out-of-the-box gamification options with many different tactics over our first year on Insided. 

First, we incentivized marking answers


Within our gamification strategy, 10 points was the right reward for marking an answer. Zapier was an easy way to make this work. 

Our Solution

  1. Zapier Zap
    1. Catch hooks by Zapier (catches the webhook event question.Answered)
    2. Insided Action Assign points to user
  2. Tell our users
    1. An announcement blog and updates to our articles about community points. 
    2. Update the system email from Insided to add the 10 point incentive to the OotB nudge to mark the answer

 

Our email 

Hi, {recipient_username}.

Your question "{question_title}" received at least one answer from our community.

If one of the answers was helpful, please mark it as solved.

1. Click the button below to view your question.
2. Select the 'solved' checkmark on the correct answer.

Marking questions as solved helps others find answers and strengthens our community knowledge base.

You also earn points for both yourself and those who help you.

  • 10 points to you for marking the answer.
  • 20 points to the Creator who provided the answer.

Next, Slack alerts to escalate unreplied questons >12 hours old to staff


While I’m a team of one, I have great support from my colleagues. And 10-20% of contributions on our community are from staff. This tactic was all about focusing their time and attention where they can make more of an impact. 

Trends showed that if a question wasn’t addressed in the first few 12-24 hours, the chances of reply decreased significantly. It just gets buried in the feed and out of sight = out of mind.

The solution itself is pretty much the same as the next one, so follow the steps below. However, we also added a cool solution of Zapier steps to pull in contextual information from Salesforce. For our staff, it helped them know if the question was from a customer/prospect/partner/colleague. All of the questions are important, but if they only have a few minutes, focus on our customers’ success. 
 

Finally, a three-pronged solution for 100% replies and bonus points


The TL;DR: 

  1. Reply to unanswered questions > 3 days old
  2. Add a moderator tag to those questions and mark them as sticky (our phrase is “Featured”)
  3. Incentivize replies and answers to those questions

Explaining the strategy

  • #3 is most important, you need to tap into the motivation. 
  • #2 is key to automating and scaling the incentive (points)
    • Sticky is also about visibility - it call out the question with the bookmark icon, label, and different color
  • #1 solves the issue of visibility. It bumps the questions to the top of the recent activity feed and also lets the user know we’re invested in getting them some help. 

Technical Solutions: 

This actually requires three separate Zaps. #1 and #2 are covered in one Zap. #3 needed separate Zaps for replies and answers. 

Zap #1 does the heavy lifting

  1. Catch Webhook for new questions (question.Asked)
  2. Delay for 72 hours
  3. Format to add weekday abbreviation 
  4. Format to next weekday
  5. Delay until next weekday at 9:01
  6. POST webhook to authenicate for steps 7 and 9)
  7. GET webhook for Insided question meta data 
  8. Filter to continue only if reply count is less than 1
  9. POST webhook for Insided to toggle sticky ON
  10. Insided Zapier Action to append moderator tag (we use “bonus”)
  11. Insided Zapier Action to create reply in Insided

Zap #2 awards bonus points for replies

  1. Insided Zapier Trigger for New Reply
  2. Filter only continue if moderator tag contains “bonus”
    • We filter out those with “closed” (see next Zap)
  3. Insided Zapier Action for assign points to user
    • We give 5 extra points

Zap #3 is a combination of the two above for awarding bonus points for answers

  1. Catch Webhook for quesiton.Answered
  2. POST webhook to authenicate for steps 3, 5, and 7)
  3. GET webhook for Insided question meta data 
  4. Filter to continue only if the moderatorTag “bonus” is found
  5. Insided Zapier Action for assign points to user
    • We give 10 extra points
  6. Insided Zapier Action for append moderator tag “closed”
    • See Zap #2 for how this filters out replies once the question has been answered
  7. POST webhook for Insided to toggle sticky OFF

 

The Results


Well, a lot of this is TBD; I just built​​​​​ it today. I’ll follow up in the replies after the first few weeks/months/quarter.

However, I do know that this solution guarantees: 

  • 100% of questions will get a reply with ~3-4 business days
  • Users will see that we’re doing what we can to bring attention to their questions

Combined with what we’ve seen from other gamification incentives and engagement trends, my hypotheses are: 

  • This will increase reply rates to help keep up increasing question volume
  • Answer rates will also increase as a lagging indicator
  • More users will earn super user status through increased engagement and new ways to earn the points required for that status

But most importantly:

  • This will make our community members feel supported
  • This will make our contributors feel more appreciated / incentivized
  • This helps us scale. No extra staff were required and it is probably just about 1-2 days work in total, most of which was just figuring this stuff out.  

What next? 


Monitor the results and iterate. I may pull in a bit from solution number 2 to add some slack alerts to engage staff on those unreplied and >3 days old questions. 

Report back with the results and learnings. 

(Oh, and I need to tell our users about the incentive and the (much simpler, for them) process of how this works.) I’ll try to remember to share that article in the replies. 

---

Shoutout: @matt enbar from Support/Professional Services really set me up for success with some help with webhooks for a related Zap a few months back. He not only helped me solve my issue, but also pointed me in the direction of learning how to do this myself (or with minimal help from my IT colleagues)


8 replies

Userlevel 3
Badge +2

@ChrisPantelis sorry for the spam 😂, but here is another one.

 

I told you @DannyPancratz knew his stuff.

Userlevel 5
Badge +4

I’m due for an update on this, so here are some details on the results: 

After a full quarter with this in place, we routinely see 95-99% of questions get a reply within a month (that’s excluding my automated replies). The tactic is working well to surface those questions that were missed a few days earlier. 

Additionally, our % of questions with a marked answer is up 20% to a whopping 75% on the quarter. Answers are a lagging indicator to replies, so this tactic surely helped. But it’s not the only reason the answers are up, as I also ramped up my moderation efforts during this quarter. That’s probbably a whole separate post, as I’m also leveraging Zapier automations for that. 🙂

Userlevel 3
Badge +2

Thanks for the update @DannyPancratz .

 

I had a 121 chat with @ChrisPantelis today and the topic of automation and the need for inspiration came up.

Userlevel 5
Badge +4

@ChrisPantelis find me on Linkedin or via Alistair if you want to chat. I’m a team of one so there’s a ton I’ve worked to automate for efficiency, but also a ton i’ve done to customize and extend beyond what’s available out-of-the-box. 

My biggest piece of advice: don’t let the predefined Zapier triggers and actions limit you. Look at the API documentation. There’s a lot more you can automate (still via Zapier, if you want) by running API operations or triggers off the webhooks. The above post is an example of that. 

Userlevel 1

Thanks for sharing what’s working in your organization @DannyPancratz ! I’m about 9 months late to this post, how is it going now?

Userlevel 5
Badge +4

@security_lion it continues to work very well. Most months we see 98 or 99% of questions getting replies. And our answer % was up over 80% for six straight months. 

I just returned from two months of leave, so it’ll be interesting to dive into if these maintained or fell (answers likely fell, as I did some additional moderation and nudging there). But I will add that having this in place made me more confident going on leave: much of the moderation to help surface unanswered questions was automated. 

Userlevel 5
Badge +3

This is fantastic, I especially like the Slack escalations! 😀

Userlevel 1

Love to see it @DannyPancratz! Welcome back, hope it was a good leave 😁

Reply