Implementing analytics in your Atlassian Marketplace app can be a complex process, but it’s essential for understanding customer behavior and enhancing your product. This guide walks you through why and how to implement analytics, the tools you’ll need, practical implementation tips, and the ethical considerations to keep in mind.
TLDR - Just give me the recommendations
Data Distribution Platform: Use Segment as the central platform for distributing your analytics data.
Back-End Implementation:
Utilize Segment’s back-end libraries to send snapshot statistics for users and groups.
Focus on capturing essential metrics that provide meaningful insights into user and group behavior.
Front-End Implementation:
Leverage the Analytics library maintained by Segment for front-end data collection.
Important: Include the following code when loading the front-end library to limit the amount of sensitive information sent to other destinations:
export const segmentClient = AnalyticsBrowser.load(
{ writeKey: "YOUR KEY" },
{
integrations: {
All: false,
"Segment.io": true
},
},
);
// Add middleware to remove referrer, search, and title fields from the context in all events segmentClient.addSourceMiddleware(({ payload, next }) => {
if (payload.obj.context && payload.obj.context.page) {
delete payload.obj.context.page.referrer;
delete payload.obj.context.page.search;
delete payload.obj.context.page.title;
}
next(payload);
});
This approach ensures robust analytics capabilities while maintaining strong privacy controls.
Why Implement Analytics?
Analytics are vital for two primary objectives:
Understanding Trial Conversion: If you offer a trial version of your product, analytics can help you understand how well trial users are engaging. Are they discovering and using the features that showcase the product’s value? This insight is crucial for converting trial users into paying customers.
Assessing Customer Health: For your existing customers, analytics help you see how they are using your product. Are they utilizing its full potential, or are there underutilized features? Understanding customer health can guide your support and development efforts, ensuring customers get the most out of your offering.
Tools of the Trade: Segment and RudderStack
To effectively implement analytics, you need the right tools. A customer data platform like Segment or RudderStack is highly recommended. These platforms serve as a central hub for your analytics data, allowing you to collect it once and then distribute it to various systems within your tech stack, such as support, communications, or marketing platforms.
Why Choose Segment? Segment is widely recognized as having defined the industry standard for data schemas—how data is collected and sent to different systems. This standardization is invaluable as it ensures consistency and reliability across all your analytics efforts. However, there are some caveats and implementations tips, which we’ll discuss shortly.
Why Choose RudderStack? RudderStack is an open-source alternative to Segment, with a paid hosted option available. It can be a more cost-effective solution, especially for teams with in-house technical expertise who can manage the infrastructure themselves. The open-source nature of RudderStack also allows for greater customization, providing more control over the data pipeline and integration processes. Additionally, by hosting RudderStack on your own servers, you gain enhanced privacy and security control, which is crucial for organizations with strict data governance requirements.
Defining and Tracking Key Events
Tracking the right events is essential for gaining meaningful insights into user behavior and engagement with your product. To do this effectively, it’s important to understand the types of calls you can make within Segment and why each one is necessary.
Types of Calls in Segment
Identify Call: This call is used to identify individual users and capture their traits, such as their name, email, or any other user-specific details. It allows you to build a profile for each user, which can then be enriched with additional data over time. The Identify call is crucial for personalizing the user experience and understanding how specific users interact with your product.
Group Call: The Group call links a user to a specific account, organization, or team, along with any relevant traits of that group. This is particularly important in B2B contexts where multiple users might belong to the same organization. By using the Group call, you can roll up individual user activities to the account level, giving you insights into how an entire team or company is using your product.
Track Call: The Track call logs specific user actions or events, such as clicking a button, completing a form, or setting up an automation rule. Each Track call is associated with a user and, if applicable, their group. This call is the backbone of event-based analytics, allowing you to capture detailed data on how users interact with your product’s features.
Combining these calls effectively helps you build a comprehensive view of user activity, both at the individual and account levels. The Identify call provides context about who is doing what, the Group call shows how these actions relate to the broader organization, and the Track call records the specific events that matter most.
Determining What Events to Track
Knowing which events to track is crucial because these actions within your product signal meaningful user engagement. The key is to focus on actions that indicate value realization and sustained use. For example, when building ProForma (acquired by Atlassian 2021), we identified and tracked the following critical events:
Creating Form Templates: Indicates that users are investing time in customizing their workflow, a strong sign of engagement.
Attaching Forms to Issues: Shows that the user is integrating your product into their core process, which is essential for long-term value realization.
Editing Forms: Continuous use and refinement of forms suggest that users find ongoing value in your product, indicating a deeper level of engagement.
Setting Up Automation Rules: Often done by power users, this action indicates a high level of engagement and a deeper understanding of the product’s capabilities.
Not All Events Are Equal
It’s important to distinguish between events that are merely informative and those that truly indicate product engagement. For instance, tracking when someone logs into your product is useful for understanding general activity, but it doesn’t necessarily indicate meaningful engagement. In the Atlassian context, actions like creating or editing a form are more valuable as they provide deeper insights into how users are actively engaging with your product’s features.
Snapshot Statistics: Tailoring Insights to Your Product
Snapshot statistics are a powerful tool in your analytics strategy, providing a high-level view of customer behavior over time. These statistics are particularly useful because they offer a more straightforward approach to understanding customer engagement, growth, and product usage without requiring complex calculations based on event data.
Why Snapshot Statistics Are Useful
Snapshot statistics should be considered from the outset when implementing analytics because they simplify the process of tracking key metrics. Instead of relying on a series of add and delete events to calculate net usage (which can be error-prone and complicated), snapshot statistics allow you to pass actual data points directly as part of the Group or Identify calls. For example, rather than trying to determine the total number of active items (such as users, templates, or automation rules) by calculating the difference between additions and deletions, you can simply pass the current number of these items directly in the call.
This method provides a clear and accurate picture of important metrics at any given time, making it easier to monitor trends and changes without having to reconstruct the data from multiple event types. These snapshot statistics can be updated regularly (e.g., nightly or weekly) to ensure your data is current and reflective of your users’ activity.
Key Considerations
Total Sums vs. Calculations: Sometimes, you’ll want to capture the total number of specific actions, such as the number of templates created. Other times, you might calculate dynamic metrics, such as the number of actions taken in the last 30 days, to track recent activity and growth trends. Snapshot statistics simplify this by allowing you to pass these totals or calculations directly, avoiding the need to derive them from raw event data.
Comparative Metrics: Comparing activity over time, such as the number of actions taken in the last 7 days versus the previous 7 days, helps highlight changes in engagement levels. Snapshot statistics make it easier to create these comparative metrics, enabling you to quickly identify shifts in user behavior.
Having these statistics readily available makes it easier to create meaningful segments or audiences, enabling you to tailor your messaging and support efforts more effectively. By integrating snapshot statistics into your analytics from the beginning, you can ensure that your insights are both actionable and easy to obtain.
Practical Implementation: Front-End vs. Back-End
When implementing analytics with Segment or RudderStack, you have the flexibility to use various libraries that can be integrated either on the front end or the back end of your application. Each approach has its own benefits and considerations, particularly within the Atlassian ecosystem.
Front-End Implementation
The front-end approach is appealing because it allows for transparency with your customers—users can see what data is being logged by inspecting browser traffic. However, this method comes with its own set of challenges, especially concerning security and privacy.
Key Considerations:
Context Data: By default, Segment captures significant context data, such as the page title, referrer, and URL, which might contain user-generated content (UGC) or sensitive information. This can pose a security risk, particularly if a JWT token is present in the URL or referrer.
Managing Segment Integrations: A Practical Tip When using Segment’s JavaScript library, be aware that certain integrations, like Intercom, will automatically load if they are enabled as destinations. This can inadvertently capture context data such as referrers, which might include sensitive information. To control which front-end libraries get loaded, you can disable all other integrations except the one you specifically need, such as Twilio Segment - Ranked #1 Customer Data Platform four years in a row per IDC (2019-2022) .
IP Address Logging: Segment automatically logs the user’s IP address. Depending on your stance on privacy and GDPR compliance, this could be an issue. One solution is to use a proxy, such as an AWS Lambda function, which receives all events and forwards them to Segment, effectively masking the user’s IP address. We can provide more details on this setup if needed.
Implementation: To avoid capturing sensitive data, you can add specific code that blocks any track or identify call from including this information:
export const segmentClient = AnalyticsBrowser.load(
{ writeKey: "YOUR KEY" },
{
integrations: {
All: false,
"Segment.io": true
},
},
);
// Add middleware to remove referrer, search, and title fields from the context in all events segmentClient.addSourceMiddleware(({ payload, next }) => {
if (payload.obj.context && payload.obj.context.page) {
delete payload.obj.context.page.referrer;
delete payload.obj.context.page.search;
delete payload.obj.context.page.title;
}
next(payload);
});
Back-End Implementation
The back-end approach offers more control, particularly in terms of security. Logging events from the back end avoids issues related to UGC or URL referrer data, making it a more straightforward option in many cases.
However, you need to consider whether you can capture all the events and interactions you need through back-end function calls alone. Some user interactions might only be observable from the front end, meaning a back-end-only approach could miss critical data points necessary for a complete analysis.
Ethical Considerations: Privacy and Transparency
Privacy: What Not to Capture
When implementing analytics, managing privacy is crucial for both legal compliance and maintaining customer trust. Here are key practices to consider:
Avoid Capturing UGC: Whenever possible, avoid capturing user-generated content (UGC). Instead of sending actual content (e.g., the title of a form), send an ID number or similar identifier to minimize the risk of exposing sensitive information.
Limit Personal Information: Avoid capturing personal information like IP addresses, email addresses, or names. Instead, use account IDs and other non-identifiable markers to achieve your goals without handling sensitive data. This approach helps protect user privacy and reduces the need for additional data management measures.
Transparency in Capturing Specific Data: In some cases, capturing specific data, such as form names, can provide valuable insights. For example, at ProForma, we chose to capture form template names (but not the contents of the forms) and were transparent with customers about this practice. This transparency is crucial for maintaining trust.
Transparency and Customer Trust
Building trust with your customers is essential, and one of the best ways to do this is through transparency. Here are some best practices:
Trust Center: Create a section in your Trust Center where you explicitly outline the types of data you collect, including examples of the payloads. This not only builds trust but also gives customers peace of mind knowing exactly what data is being collected and how it’s used.
Privacy Policy: Keep your privacy policy up to date, clearly stating where data is sent and how it is processed. Be thoughtful about which applications you connect to Segment or RudderStack as destinations for your analytics data, and ensure these choices are reflected in your privacy policy.
Giving Customers Flexibility: The Option to Opt-Out
Providing an option to opt out of analytics and data capture is a critical aspect of building trust, but it’s important to implement this feature thoughtfully, considering both product-level control and the different phases of user engagement.
Product-Level Control: The opt-out option should be offered at the product level, allowing Jira Administrators to disable analytics and data capture across the entire product. In ProForma, for example, if a Jira Admin disabled analytics, it also disabled messaging platforms like Beamer and Intercom, effectively cutting off all analytics and data capture related to user activity. This approach ensures that organizations have full control over their data privacy settings.
Free Versions and Trial Periods: In some cases, it may be beneficial to restrict the opt-out option during certain phases. For example, in the free version of ProForma, we did not provide the option to disable analytics. This allowed us to better understand how users were interacting with the free product and to communicate the value proposition of upgrading to a paid version. Anecdotally, we believe this approach contributed to driving upgrades, as it helped us demonstrate the product’s value more effectively.
Similarly, during trial periods, it might be advisable to keep analytics enabled by default. Analytics can play a crucial role in helping users reach the “aha moment” by allowing you to guide them through the product’s key features and demonstrate its value. Once the user transitions to a paid plan, you can then offer the option to disable analytics if privacy becomes a concern.
Why This Matters: Allowing product-level opt-out respects customer privacy and builds trust, particularly for organizations with strict data governance policies. However, strategically managing when and how this option is offered—such as during free trials or in free versions—can help you maximize user engagement and conversion by leveraging analytics to highlight the product’s value.
So What? Turning Data Into Action
Once you’ve implemented analytics and started capturing data, the next step is turning that data into actionable insights that drive your product forward.
Key Actions to Consider:
Feature Usage Analysis: Use your analytics to understand which features are most and least popular. This can inform your development priorities, marketing efforts, and customer support strategies.
Customer Segmentation: With the data you’ve collected, create meaningful segments of your customer base. This allows you to tailor communications, offers, and support to different groups, enhancing their experience and increasing engagement.
Product Improvement: Identify pain points or underused features through your data analysis. This can guide your product roadmap and help you focus on the areas that will have the most significant impact on customer satisfaction and retention.
Conclusion
Implementing analytics in an Atlassian Marketplace app is not just about collecting data—it’s about using that data to make informed decisions that enhance your product and customer experience. By focusing on the right metrics, maintaining privacy, and being transparent, you can leverage analytics to drive meaningful improvements, build stronger customer relationships, and ensure compliance with data protection regulations.
Other reference:
Segment Quick Start Guide for Accoil Analytics