Livedocs Document Basics

Livedocs empowers you to explore, analyze, and share data in a collaborative, dynamic environment. This guide provides a foundational understanding of the key concepts and workflows you need to get started with Livedocs.

In this guide, you will learn:

• How to create and manage Livedocs documents.
• The basics of data pipelining to connect and transform your data.
• An introduction to automation and scheduled runs for keeping your workflows up-to-date.
• How Sidekick supports you in managing data sources, secrets, and scheduled runs.

This guide is designed for users who have completed the Creating Your First Livedoc tutorial and are ready to deepen their understanding of Livedocs' capabilities. It is an essential resource for anyone looking to master the basics before diving into advanced features and workflows.

note

Prefer to dive in without completing the tutorial? That is okay, too! This basics guide is designed to be comprehensive and accessible, even for first-time users who want to explore Livedocs at their own pace.

Creating a Document

To begin working in Livedocs, you first need to create a new document. Creating a document is the starting point for integrating data and developing insights in a collaborative environment.

Steps to Create a Document

1. Navigate to the Docs tab in the top navigation bar.
2. Click the Create doc button at the top-right corner of the interface. This action opens a blank document in the workspace.
3. In the blank document's canvas, modify the default title provided (New doc - [current date]) to reflect the purpose or content of your document. The title is formatted in bold for emphasis and is directly editable.
4. Add an optional description for your document in the smaller description field below the title. This field features placeholder text that reads: "Describe your doc. Mention key insights, data sources, or any interesting findings... ." The description helps provide context for collaborators.

Your new document is now ready for you to add elements (more on this later) and begin building your analysis. Each document in Livedocs is automatically saved as you work, so you can focus on your tasks without worrying about data loss.

Before diving into manual element creation, explore Livedocs' quick-start shortcuts. These tools let you rapidly set up preconfigured elements and data sources, helping you start your analysis in seconds.

Exploring Quick-Start Shortcuts

When you create a new document or clear all elements from an existing one, Livedocs displays a set of shortcuts in the workspace. These shortcuts are designed to help you quickly start building your document with preconfigured elements and data source options:

• Feature Cards: Quickly add elements prepopulated with SQL or Python code for common tasks like querying data, writing back to a database, or fetching data from an API.
• Database Connection: A button that provides quick access to the Create database connector dialog box, enabling you to connect to a PostgreSQL database without navigating away.
• File Upload Area: Quickly add file-based data sources, such as CSV or Excel files, by dragging and dropping them into the workspace or selecting them via a file dialog box.

These shortcuts streamline your workflow, offering immediate access to essential tools and configurations when starting fresh in your document workspace.

These shortcuts are just one way to begin building your document. For greater control and flexibility, you can manually add and configure elements to create a customized workflow.

Setting Up Data Sources

A document in Livedocs relies on data sources to fuel its analyses, visualizations, and workflows. Before adding elements like Query or Chart, ensure you have the required data connected to your document.

note

For detailed instructions on adding data sources, refer to the guides in our Managing Data Sources section:

• Connecting to a PostgreSQL Database: Step-by-step instructions for setting up a database connection.
• Importing Data from CSV Files: Learn how to upload and use CSV data in your document.

There are several ways to set up data sources in Livedocs:

• Quick-Start Shortcuts: Use the + Connect button in the document workspace to quickly configure a database connection. For file-based sources, use the file upload area to browse for files in your system using a dialog box or drag and drop them to the upload hotspot.
• File Upload Area: Easily upload file-based data sources like CSV or Excel files by browsing for files or dragging and dropping them into the workspace.
• Data Tab: Navigate to the Data tab in the top navigation bar to manage your data sources comprehensively. This tab provides options for connecting databases, uploading files, and viewing existing data connections.

tip

Need to work with an API? Use a Python element to fetch, process, and integrate data from external APIs directly into your document.

By setting up your data sources through these options, you can seamlessly integrate external data into your document and start building powerful, data-driven workflows.

With your data sources ready, you can now start adding elements to your document. Elements are the core of your analysis, enabling you to query, visualize, and manipulate your data dynamically.

Adding Elements to Your Document

Elements are the building blocks of a Livedocs document, enabling you to create interactive and data-driven narratives. Each element serves a specific purpose, from visualizing data to documenting findings or performing complex transformations.

Types of Elements

Livedocs provides five types of elements, each designed to support various aspects of your analysis:

1. Chart: Visualize data using customizable charts to identify patterns, trends, and insights.
2. Query: Dynamically retrieve data from connected sources using SQL.
3. Python: Write Python code to transform data, run calculations, or create custom outputs.
4. Text: Add notes, annotations, or detailed explanations to complement your analysis.
5. Table: Present structured data in a tabular format from files, database tables, or Polars DataFrame objects.

Steps to Add an Element

1. Locate the elements toolbar at the bottom center of the document workspace. Use this toolbar to add a new element to the end of your document.
2. To insert an element between two existing ones, hover over the gap between them until the "+" icon appears. Clicking the icon will open a small version of the elements toolbar.
3. Choose the element type you want to add (e.g., Chart, Query, Python).
4. Configure the element once it appears in the document:
   • For Query or Table, select a data source from the dropdown menu.
   • For Chart, choose both a data source and a chart type.
   • For Python, write or paste the code you want to execute.
   • For Text, type your content directly into the editor.

Once you have added and configured elements, the next step is to make use of their common actions to manage and refine your document effectively.

Common Actions with Elements

Managing elements in Livedocs involves several common actions that help you organize and refine your document. The markers in the annotated screenshot below correspond to the headings in this section, showing where each action can be performed in the document workspace.

note

Most of the common actions listed below are supported by all five Livedocs elements. However, some actions are element-specific and noted accordingly.

A. Reordering

Use the drag handle at the top-left corner of an element to move it to a new position within the document. This allows you to structure your document logically and efficiently.

B. Collapsing or Expanding

Most elements, except Table elements, have a chevron icon located near the left corner of their container, just below the drag handle. Clicking the chevron toggles between collapsing and expanding the element's content (e.g., editor, query results, or charts).

Collapsing elements reduces workspace clutter, helping you focus on active sections.

• Not Collapsible: Table elements do not support collapsing or expanding.
• Special Behavior:
  • Query and Python elements allow independent collapsing and expanding of both their editor and results areas.
  • Text elements only allow the results area to collapse, while the editor remains visible at all times.

C. Naming

Give each element a descriptive name to clarify its purpose. Naming is especially helpful for Query and Python elements because these elements often generate data variables used elsewhere in your document.

• Not Nameable: Table and Text elements.

D. Specifying a Data Source

For Chart, Query, and Table elements, bind the element to a relevant data source by selecting an option from the data source dropdown menu. This ensures the element uses the correct dataset for its operation.

Clicking the data source dropdown reveals a list of configured data sources, a search field for finding specific sources, and a Connect a data source option. Selecting this option redirects you to the Data tab to add new data sources.

• Unsupported: Python and Text elements.

E. Running

Execute an individual element by clicking its Run button (a play icon). Running elements individually is useful for testing and troubleshooting specific sections without executing the entire document.

The behavior of the Run action varies depending on the type of element:

• Query: Executes the SQL query and retrieves data from the connected source. The query results are displayed in the results area of the element.
• Python: Runs the Python code within the element, performing any transformations, calculations, or operations specified in the script. Outputs are displayed in the Python element's output area.
• Chart: Re-renders the chart using the most recent data from its connected source or calculated variable. Chart-related customizations, such as chart type, colors, aggregations, and axis settings, also impact how the chart is displayed after running.
• Table: Refreshes the displayed data based on updates to its connected source or data pipeline.
• Text: Re-renders the rich text content inside the element in its results area. While the impact may not be immediately noticeable, running a Text element ensures that any updates or formatting changes, such as hyperlinks or styling, are correctly rendered.

tip

For elements that are dependent on others (e.g., a Chart element using data from a Query), ensure that the upstream elements are run first to avoid errors or outdated results.

F. Accessing Other Actions (Deleting, Embedding)

Open the three-dot menu (ellipsis icon) near the top-right corner of an element's container to access additional actions, such as deleting the element or embedding it in websites.

If you choose to embed an element:

• Copy the provided <iframe> embed code to add it to websites.
• Use the embeddable link for supported apps like Notion or Coda.

note

The three-dot menu is positioned outside an element's main container to avoid overlap with content. The menu appears every time your mouse pointer enters the element's container.

G. Inserting

All elements have a "+" icon at the bottom of their container. Hover over this icon and click it to reveal a smaller version of the elements toolbar where you can choose the element to add.

If the current element has no succeeding one, the new element is placed after the current. If there is an existing one after the current element, the new element is inserted between the current and succeeding ones.

Inserting is particularly useful when managing complex documents with many elements. This approach saves time by avoiding the need to reorder existing elements manually.

---

By mastering these common actions, you can efficiently organize and enhance your Livedocs document, ensuring it remains structured, dynamic, and ready for collaboration.

As you progress, you will often find a need to connect data across elements, enabling seamless workflows and in-depth analysis. This is where data pipelining becomes crucial, allowing you to link elements like Query, Python, and Chart to transform and share data dynamically.

Introducing Data Pipelining

Data pipelining in Livedocs is the process of connecting, reusing, and transforming data across elements to create seamless workflows. By understanding how variables work and how elements interact, you can build efficient, dynamic documents that fully leverage your data sources and analytical capabilities.

Understanding Variables in Livedocs

Livedocs utilizes two types of variables that play crucial roles in data pipelining. These two are shown in action in the screenshot below, annotated and explained in the following list.

note

Refer to the numbered annotations in the screenshot to understand where these variables appear and how they are used in the workspace.

1. Polars DataFrame Variables: These variables are automatically generated by Query elements and stored as Polars DataFrame objects. They are compatible with elements like Chart, Table, and even other Query elements, allowing data to flow dynamically across your document. Query elements can use DataFrame objects from previous Query elements as their data source, enabling a layered and iterative approach to data manipulation. These variables follow a naming convention (dataframe_N), where N is an incremental number starting from 0.
2. Python Variables: These are standard Python variables, such as strings, lists, or dictionaries. They are globally accessible across Python elements within the same document but cannot be used as data sources in other element types.

Grasping the differences between these variables is essential for creating effective pipelines that seamlessly connect and transform your data.

Example: Sharing Python Variables Across Python Elements

Python variables are versatile for intermediate data transformations and sharing across Python elements. Here is a basic example:

Python Element 1:

# Define a Python variable
team_members = ["Alice", "Bob", "Charlie"]

Python Element 2:

# Access the variable defined in Python Element 1
print(f"Team members: {team_members}")

This demonstrates how Python elements can share variables, enabling efficient and reusable workflows.

How Data Pipelining Works

Data pipelining connects elements like Query, Python, Chart, and Table to create a flow of data, allowing dynamic and modular workflows. Here is how it works:

• Pipeline Flow: Data flows from one element to the next. For example, a Query element retrieves data, which can then be transformed in a Python element and visualized in a Chart or Table.
• Stacking and Narrowing Data: Successive Query elements can refine or narrow data dynamically. For instance, a Query element can use a dataframe_N variable from a previous Query to filter results further.
• Manipulating Data in Python: Python elements allow for advanced data manipulation. Modified or newly created DataFrame objects can be passed to downstream elements for visualization or further analysis.
• UI Feedback: At the bottom of Query and Python elements, Livedocs displays tags showing dataframe_N variables and their usage counts across the document (e.g., dataframe_0 | 2). These tags provide quick insights into data dependencies and how often a variable is used, helping you maintain control over your document's pipeline.

Example: Query, Python, and Table Elements in a Pipeline

Here is how data flows through a basic pipeline:

1. Query Element: Generate data from a connected source.

   SELECT Age, DailyRate, EmployeeNumber 
   FROM employee.csv
   WHERE EmployeeNumber < 10
   LIMIT 3;
   • Creates dataframe_0 containing the query results.

2. Python Element: Transform the data in dataframe_0.

   dataframe_0 = dataframe_0.with_columns(
      (dataframe_0["DailyRate"] * 1.1).alias("UpdatedRate")
   )
   • Adds a new column UpdatedRate reflecting a 10% increase in DailyRate.

3. Table Element: Display the transformed data.

   • Use dataframe_0 as the data source for visualization.

tip

Chart elements can also utilize modified DataFrame variables, providing dynamic visualizations for your pipeline.

This pipeline demonstrates the flow of data from a Query to a Python transformation and finally into a Table for presentation.

With this understanding of how elements interact, let us explore some practical scenarios where data pipelining truly shines.

Examples of Practical Use

Data pipelining is not just about linking elements; it empowers you to build workflows that align with your specific needs and goals. Here are some use cases:

1. Visualizing Filtered Data: Use a Query element to fetch raw data from a database, process it in Python to apply filters, and then visualize the filtered results in a Chart element.
2. Dynamic Analysis: Combine multiple Query results in a Python element to perform advanced analytics, such as aggregations or correlations, and display the outcomes in a Table or Chart element.
3. Scenario Planning: Create multiple data pipelines for what-if scenarios, like best-case vs. worst-case forecasts, and switch between them dynamically.

tip

To ensure smooth workflows, name your variables descriptively and document your pipelines within your Livedocs document using Text elements.

Key Takeaways

• Reusable Variables: Python variables are shared across Python elements, while Polars DataFrame variables can serve as data sources for multiple compatible elements.
• Seamless Data Flow: By connecting elements like Query, Python, and Chart, you create a pipeline that dynamically handles data transformations and visualizations.
• Iterative Data Refinement: Successive Query elements can build upon previous results, enabling layered transformations and precise data preparation.
• Efficient Workflows: Pipelines eliminate the need for repetitive data operations by reusing variables and maintaining data consistency across elements. For example, you can reuse a Polars DataFrame variable in multiple Chart or Table elements without duplicating queries or manual data updates.

---

Now that you understand the concept of data pipelining, it is time to explore how Sidekick enhances your workflow. Sidekick provides quick access to data sources, variables, and secrets, making it an essential tool for effective document management. Head to the next section to learn more about its features.

Using Sidekick

Sidekick is your indispensable assistant in Livedocs, providing quick access to tools for exploring data sources, managing secrets, monitoring variables, and scheduling document runs. Positioned on the left side of the workspace, Sidekick helps streamline your workflow by centralizing these critical features in one convenient location.

info

This guide briefly introduces Sidekick to raise awareness about its role in your document workflows. For a detailed explanation of all its features and functionality, refer to the Understanding Sidekick guide. Keeping one source of truth ensures consistent, in-depth coverage.

Sidekick at a Glance

Sidekick simplifies and enhances your document-building process with the following capabilities:

• Exploring Data Sources: Quickly browse and inspect schemas to identify the data you need while building a Query or Chart element.
• Managing Secrets: Securely access and apply sensitive credentials like API keys, ensuring data protection without workflow interruptions.
• Monitoring Variables: Dynamically track variables created or used in your document. This feature is particularly useful for managing data pipelining across elements and debugging workflows.
• Scheduling Document Runs: Automate document updates by scheduling runs directly from Sidekick, ensuring your data and analyses stay current.

For instance, when working with a complex document involving multiple data sources, Sidekick provides a quick overview of available variables and allows you to manage connections seamlessly, all without leaving the workspace.

Next Steps

Mastering Sidekick will significantly enhance your efficiency in Livedocs. As you build more complex documents, its features will become indispensable for seamless data integration, analysis, and automation. Explore the next section to learn about managing your documents, including running, sharing, and publishing.

Managing Your Document

Efficiently managing your Livedocs document ensures that your data and analyses remain accurate, collaborative, and ready for presentation. This section guides you through essential actions, such as running your document to refresh data, sharing it with your team, and publishing updates to make your work accessible to others.

Running the Entire Document

In Livedocs, running a document ensures that all elements are processed with the latest data and configurations. This is particularly useful when working with dynamic data sources or when validating the results of your document before sharing or publishing.

How to Run a Document

1. In an open document, locate the Run button at the top-right corner of its workspace.
2. Click Run to execute all elements in the document sequentially.

Livedocs processes each element, fetching data from sources, executing code, or rendering charts, as applicable. Once the document finishes running, the elements will display their respective results. A progress indicator within the Run button provides feedback during execution.

tip

If your document contains time-intensive elements (e.g., large datasets or complex Python code), consider running specific elements individually using their respective Run button to save time.

Benefits of Running a Document

• Ensures Fresh Data: Running the document refreshes data pulled from dynamic sources like APIs or databases.
• Validates Analysis: By executing all elements, you can confirm the accuracy and integrity of your calculations, queries, and visualizations.
• Prepares for Sharing or Publishing: Running the document ensures that collaborators or readers will see the most up-to-date and accurate information.

Once your document is refreshed and validated, you can move forward to sharing it with your collaborators.

Sharing and Collaboration

Sharing a document in Livedocs allows you to collaborate with others by granting them access to view or contribute. However, before sharing, you must publish the document. Publishing makes the document publicly accessible and updates the version seen by others. Without publishing, the document cannot be shared effectively.

How to Share a Document

1. Publish the Document:
   • Click Publish Changes at the top-right corner of the workspace to mark the document as public. This step is required to enable sharing options.
   • Once published, an alert message will confirm that the document is now public and shareable.
2. Access the Share Menu:
   • Click the Share button to open the sharing menu.
   • Toggle Share with link ON to allow anyone with the link to view the document. If toggled OFF, access will be restricted.
3. Use Sharing Options:
   • Copy Link: Copies the shareable URL to the clipboard for distribution.
   • Embed Doc: Opens a dialog with an iframe embed code and an embeddable link for use in third-party apps like Notion or Coda.
   • Send doc by email: Enter the recipient's email address in the provided field and click Send to invite them.

note

The Share with link toggle option becomes available only after the document is published. Unpublished documents cannot be accessed by anyone, even if the link is shared.

Benefits of Sharing

• Streamlined Collaboration: Collaborators can view or contribute to the document in real time, reducing the need for offline edits or version control issues.
• Flexible Access Options: Share links, embeds, or emails to accommodate various collaboration needs.
• Enhanced Visibility: Publishing ensures that the latest updates are visible to stakeholders or collaborators, maintaining transparency and consistency.

After sharing your document, it is essential to understand how publishing updates work to keep your content accurate and aligned with your audience's expectations.

Publishing Changes

Publishing in Livedocs is more than just publicizing a document; it is also a way to save and update the shared version. Without publishing, changes made to a document remain private and are not reflected in the shared or public view.

Why Publish Your Document?

• Control Visibility: Publishing ensures that your document is accessible to others. Without publishing, sharing options like Share with link will not function.
• Update Shared Versions: Any changes made to the document are visible only after publishing, keeping the public or shared view in sync with your updates.
• Prepare for Collaboration: Publishing enables sharing features and communicates that the document is ready for review or presentation.

How to Publish Changes

1. Click the Publish Changes button at the top-right corner of the workspace.
2. An alert message will confirm that the document is now public and updates are applied.

Key Insights About Publishing

• Updates Made Public: Changes to a document are not visible to others until published. This ensures that only finalized edits are shared.
• Linked Sharing: Publishing is required before enabling Share with link, ensuring visibility to the intended audience.
• Document Embedding: Once published, the document can be embedded using the provided iframe code or specialized embed link.

tip

Use publishing strategically to finalize updates, ensuring stakeholders always view the most accurate and up-to-date version of your document.

---

By mastering document management in Livedocs, you can efficiently collaborate, keep your documents updated with fresh data, and present your findings with confidence. Next, we will explore how automation and scheduling can further streamline your workflows, making Livedocs an even more powerful tool for your projects.

Briefing on Automation

Automation in Livedocs enables you to schedule document runs, ensuring your workflows remain up-to-date with minimal manual intervention. By using scheduled runs, your document can automatically refresh data, perform analyses, and provide stakeholders with the most accurate and current information.

Scheduled Runs Overview

Scheduled runs are ideal for:

• Data Refreshes: Automatically pull the latest data from dynamic sources like APIs or databases.
• Routine Updates: Execute recurring analyses without manual input.
• Stakeholder Communication: Ensure shared documents always reflect the latest results, minimizing the risk of outdated information.

Here is a glimpse of the UI in Livedocs for managing scheduled runs. The Schedules section in Sidekick provides a clear overview of all your scheduled workflows, making it easy to configure and monitor them:

The Schedules section in Sidekick, displaying recently completed and upcoming document runs.

tip

Schedule runs to streamline repetitive tasks, such as daily summaries or weekly performance reports, saving time and effort while maintaining data accuracy.

Notifications and Alerts

Livedocs provides notification options to keep you informed about scheduled run activities:

• Success Alerts: Confirm when a document run completes successfully.
• Failure Alerts: Notify you of issues, such as broken data connections or errors during execution.

You can enable notifications through your account settings, either via email or in-app alerts, for real-time updates on scheduled workflows.

---

This section introduces the basics of automation and scheduled runs, giving you a foundational understanding of how they work in Livedocs. To truly unlock the full potential of this feature, including step-by-step guidance on configuration and troubleshooting, refer to our Scheduling Document Runs guide in the advanced documentation.

note

Remember, we are here to support your journey as you master Livedocs. Take your time exploring these concepts, and when you are ready, dive deeper into the advanced guides for more hands-on learning.

With this introduction to automation, you are now ready to reflect on the foundational skills covered in this guide and explore the next steps in mastering Livedocs.

Recap and Next Steps

This guide has provided you with a foundational understanding of working with Livedocs. You now have the knowledge to perform essential tasks and take the first steps toward advanced features:

• Create and Manage Documents: Set up data sources, add elements, and organize your document for effective analysis.
• Utilize Data Pipelining: Connect and transform data across elements, creating dynamic and reusable workflows.
• Explore Sidekick: Familiarize yourself with its centralized tools for managing data sources, secrets, and scheduled runs to enhance your workflows.
• Understand Automation Basics: Understand how scheduled document runs automate updates and keep your workflows current.

To further develop your skills, explore these additional guides in our documentation:

• Understanding Sidekick: Learn about Sidekick's features in detail to maximize your productivity.
• Scheduling Document Runs: Step-by-step instructions for configuring and managing automated workflows.
• Best Practices for Document Management: Tips and strategies for maintaining organized, efficient, and collaborative documents.

By building on the concepts covered here, you will be well-equipped to create, manage, and share powerful data-driven documents in Livedocs. As you progress, revisit this guide whenever you need to refresh your understanding of the basics or lay a stronger foundation for exploring complex workflows and capabilities.