Unboxing ‘Out-of-the-Box’ Analytics: Why Your Team Needs Data Literacy

Any software vendor looking to compete in today’s higher education space invariably offers “out-of-the-box” (OOTB) analytics: prebuilt dashboards and reports promising quick insights into performance, user engagement, and other metrics. These tools can be appealing, especially for institutions with limited resources or expertise in analytics. However, OOTB solutions often fail to capture the full complexity of user experiences, potentially resulting in incomplete—or even misleading—data interpretations. Institutions seeking to bridge this gap may be advised to turn to an analytics office (if one exists) or invest in costly custom solutions, both unrealistic options for resource-constrained schools. In this session, we propose an alternative: do not completely eschew OOTB analytics, but equip your team with the skills to approach these tools critically. The ideal staff member to undertake this role combines institutional insight with foundational data literacy—a modest skill set that can be cultivated within existing staff. We will share examples of metrics and visualizations from OOTB analytics that have been misleading, along with the competencies our team developed to uncover discrepancies and probe deeper into the data.