DataLogger

Raisin provides a buffered data logger with CSV export support.

The global logger is accessed via raisin::DataLogger::getInstance(). Dynamically loaded controllers and plugins should use their injected dataLogger_ instead of the global singleton.

Creating a log directory

Initialize the logger once with createALogFile(deviceType, directory):

raisin::DataLogger::getInstance().createALogFile(
  "raibo2", raisin::getLogDirectory("raisin_data_logger"));

This creates a directory named <deviceType>_YYYY-mm-dd-hh-mm-ss under the provided base directory and writes log_0.raisin_data (rotating across up to 4 files as size grows). In raisin_raibo2_node this call is already handled at startup, so you should not call it again from plugins or controllers.

Logging data groups

Logging starts by declaring a data group with initializeAnotherDataGroup. The ownersName becomes the CSV file prefix, and the args are pairs of std::string label and data values. Supported data types include:

  • Eigen::Matrix<T, S1, S2>: A matrix type from the Eigen library, useful for handling mathematical matrices with specific dimensions.

  • std::vector<T>: A standard vector, which can contain elements of any type T.

  • raisim::VecDyn: A dynamic vector type specific to the Raisim library, typically used in robotics and simulation.

  • raisim::Mat<n, m>: A matrix type from the Raisim library, defined by its dimensions n and m.

  • T: A generic placeholder representing a variable of any basic data type (e.g., int, float, double).

The method returns a size_t logIdx that identifies the group. Use the same order and shapes when calling append; otherwise the reader will warn about mismatched lengths.

For vectors and matrices, the values must be initialized before calling initializeAnotherDataGroup because the logger captures the shape in the label row.

To complete the logging process and export the data to a CSV file, the following steps should be taken:

raisin::DataLogger::getInstance().flush();
raisin::DataReader file(raisin::DataLogger::getInstance().getDataDirectory());
file.exportToCsv();

These steps are also managed within raisin_raibo2_node, so there is no need to repeat them in other parts of the application. exportToCsv writes <ownersName>_<index>.csv files into the same log directory.

To control rotation size, call setAllowedDataSize (bytes). raisin_raibo2_node sets this from max_logging_volume (MB).

For data visualization, you can use tools like PlotJuggler, as detailed in the operation manual.

For practical implementation examples, refer to the test_raisin_data_logger.cpp.

// Copyright (c) 2020 Robotics and Artificial Intelligence Lab, KAIST
//
// Any unauthorized copying, alteration, distribution, transmission,
// performance, display or use of this material is prohibited.
//
// All rights reserved.

// gtest
#include <gtest/gtest.h>

// std
#include <string>
#include <vector>

// ros

// raisin
#include "../include/raisin_data_logger/raisin_data_logger.hpp"

//
// Tests
//

TEST(ParameterTest, ParameterTest1) {
  /// initialize logging variables
  double dDouble1 = 1.5, dDouble2 = 2.5;
  float dFloat1 = 2.3f, dFloat2 = 4.2f;
  bool dBool1 = true, dBool2 = false;
  std::string dString1 = "hello", dString2 = "hi";
  Eigen::MatrixXd dEigen1(20, 10);
  Eigen::MatrixXf dEigen2(3, 2);
  Eigen::Matrix3f dEigen3;
  dEigen1.setConstant(2);
  dEigen2.setConstant(2);
  dEigen3.setConstant(3);
  std::vector<double> dSV = {1, 1, 1};
  std::vector<float> fFV = {2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2};
  raisim::Mat<3, 3> rotMat;
  raisim::VecDyn gc(6);
  gc.e() << 0, 1, 2, 3, 4, 5;
  rotMat.setIdentity();

  /// create this log file one in the main function
  raisin::DataLogger::getInstance().createALogFile("test", "/tmp");

  /// initialize a set of
  size_t logIdx1 = raisin::DataLogger::getInstance().initializeAnotherDataGroup(
    "raisin_datalogger_test", "life", dDouble1, "dead", dBool1, "killed", dBool2, "name", dString1,
    "name2", dString2, "mana", dFloat2, "massMat", dEigen1, "bias", dEigen3, "coolTime", dFloat1,
    "dSV", dSV, "fFV", fFV, "gc", gc, "rotMat", rotMat);

  /// initialize a set of
  size_t logIdx2 = raisin::DataLogger::getInstance().initializeAnotherDataGroup(
    "raisin_datalogger_test2", "life", dDouble1, "dead", dBool1, "killed", dBool2);

  raisin::DataLogger::getInstance().setAllowedDataSize(20000000);

  for (int i = 0; i < 71300; i++) {
    auto start = std::chrono::system_clock::now();
    raisin::DataLogger::getInstance().append(logIdx1, dDouble1, dBool1, dBool2, dString1, dString2,
      dFloat2, dEigen1, dEigen3, dFloat1, dSV, fFV, gc, rotMat);

    if (i % 2 == 0) {
      raisin::DataLogger::getInstance().append(logIdx2, dDouble1, dBool1, dBool2);
    }

    auto end = std::chrono::system_clock::now();
    auto elapsedTime = (end - start).count();
    usleep(200);
    // std::cout<<"elapsed time is " << elapsedTime<<std::endl;
    RSFATAL_IF(
      elapsedTime > 1000000, "took too long to save. ellapsed time (us) is " << elapsedTime)
  }

  raisin::DataLogger::getInstance().flush();
  RSINFO("reading " << raisin::DataLogger::getInstance().getDataDirectory())
  raisin::DataReader file(raisin::DataLogger::getInstance().getDataDirectory());

  auto& dataSet = file.getDataSet("raisin_datalogger_test");
  auto data = dataSet.data;

  auto& dataSet2 = file.getDataSet("raisin_datalogger_test2");
  auto data2 = dataSet2.data;

  EXPECT_EQ(1.5, std::stod(data["life"][0]));
  EXPECT_EQ(true, bool(std::stod(data["dead"][0])));
  EXPECT_EQ(false, bool(std::stod(data["killed"][2])));
  EXPECT_EQ(4.2, std::stod(data["mana"][0]));
  EXPECT_EQ(2.0, std::stod(data["massMat_2_3"][6]));
  EXPECT_EQ(3., std::stod(data["bias_2_1"][4]));
  EXPECT_EQ(2.3, std::stod(data["coolTime"][3]));
  EXPECT_EQ("hello", data["name"][5]);
  EXPECT_EQ(1., std::stod(data["dSV_2"][3]));
  EXPECT_EQ(2., std::stod(data["fFV_4"][1]));
  EXPECT_EQ(1., std::stod(data["rotMat_2_2"][1]));
  EXPECT_EQ(3., std::stod(data["gc_3"][1]));

  EXPECT_EQ(1.5, std::stod(data2["life"][0]));

  file.exportToCsv();
}

API

class DataLogger

Public Functions

void createALogFile(const std::string &deviceType, const std::string &directory)

create a log file in a given directory

Parameters:

directory – to create a log file

inline const std::string &getDataDirectory()
Returns:

directory where log file is saved

template<typename ...Args>
inline size_t initializeAnotherDataGroup(const std::string &ownersName, Args... args)
Parameters:

  • ownersName – owner of the log

  • args – pairs of (data name, data variable)

Returns:

index of log counter. this should be used in append

template<typename ...Args>
inline void append(size_t logIdx, Args... args)
Parameters:

  • logIdx – index of log counter.

  • args – data variables

void flush()

flush data

inline void setAllowedDataSize(size_t allowedDataSize)

set the allowed data size

Parameters:

allowedDataSize – (in byte)

class DataReader
struct LoggingSet