Writing software that efficiently utilizes the vector units of RISC-V CPUs without expert knowledge requires programmers to rely on compiler autovectorization or hand-crafted libraries. However, smarter approaches—such as autotuning frameworks—have lacked integration with the RISC-V RVV extension, significantly limiting the efficient deployment of complex AI workloads.

In this presentation, we introduce a workflow based on the TVM compiler to efficiently map AI workloads onto RISC-V vector units. Rather than relying on hand-crafted libraries, we integrated the RVV extension into TVM's MetaSchedule framework, a probabilistic program framework for tensor operation tuning.

We implemented various RISC-V SoCs on an FPGA and tuned a wide range of AI workloads on them. Our resulting code exhibits lower latency and a smaller memory footprint compared to code generated using GCC's compiler autovectorization. We are currently in the process of open-sourcing this solution.