Setting the D5M Terasic Camera using Nios II at 1920x1080

Written by Holguer Andres


  • Quartus Version Above or equal to 14.1
  • D5M Camera.
  • RAM (Where we can store the video acquisition).
  • HDMI/VGA/SDI, or others video outputs (to stream out the video in order to see the results).

This manual is composed by two sections:

  • Hardware description part.
    n this part you can learn how to create a video acquisition system using Qsys.
  • Software part.
    n this part you will learn how to communicate with the D5M camera using Nios II through the "Audio and Video config" core.


  1. Depending on the Quartus Version that you might have, you should download from the following link the “Altera University Program IP Cores”.
    *note: If you are using a version above 16.1 you don’t need to install this.
  2. Once you have downloaded the version according to your Quartus version, you must install it on the root directory according the version of Quartus.
  3. The core we are going to work with is the “Audio and video config core”, therefore it would be useful to download the PDF manual that is located on the same page from where you have downloaded the Installer of the Altera university program.
  4. Now depending on the board you are using, you should use the “System builder” to create the initial template. However for sake of time, in this manual we are going to use the Cyclone V GX starter kit to illustrate the basic steps(Generated file).
  5. After generating the basic template for this exercise, open Quartus, and open the template.

  6. Now download the following Qsys template (basic_nios.qsys), or use your own in which you have a Nios II.
  7. Copy this file to the directory of the template.

  8. Open Qsys, and open the template file<basic_nios.qsys> or you qsys file in which you will find the following:

  9. Now for the video acquisition it is vital to have a ram in which we can store the video acquisition coming from the camera. As the Cyclone V GX Starter Kit has a LPDDR2 ram we are going to add this ram controller to this Qsys file. Nevertheless, it is important to know that the RAM controller specifications for the video acquisition may vary depending on the board you are using; therefore, you might have to change this one to port the qsys system to other board of your interest.

    So let’s start by searching on the IP library the LPDDR2 SDRAM Controller, and add it.

  10. Once it is added, you have to configure the RAM controller with the following settings:

  11. Once you have configured the RAM controller, you must connect it as follows:

  12. Download the following subQsys System "video_d5m_input.qsys", and paste it on the project directory, which contains all the necessary cores for the camera acquisition.
  13. Save the project.
  14. Now go to the menu "File->Refresh System"
  15. After refreshing the system, you can search on the IP library the Subsytem with the name "video_d5m_input5", add it, and rename it with the name "camera"

  16. If you want to explore this subsystem you can press right click on "camera", and press "Drill into system", and to go back to the top system you can press the highlighted button.

    as you can see this subsystem has the core "Audio and Video Config", which is used to configure the D5M through I2C, you can double click to explore it, and you will find the following configurations (in the section software, these configurations are properly explained):

  17. Now it is time to add a “clocked video output” , and configure it as follows:

  18. Now it is time to interconnect all the modules we have added to the qsys design as follows:
  19. Once everything is connected, we are going to download another Qsys Subsytem "plls_camera_hdmi.qsys", which contains 2 PLLs, one for the camera which has to work at 25MHz, and the other one for the HDMI interface that works at 148.5 MHz.
    You have to copy this subsystem in the directory of the project.
  20. Save the Qsys project.
  21. Press F5 to refresh the System.
  22. Now you can search the module called "plls_camera_hdmi", add it to the Qsys system, and rename it with the name "video_clocks"

    as the subsytem "video_d5m_input", you can drill into this module to explore what is inside it.

  23. Connect the added "video_clocks" as follows
  24. Now that the Qsys design has been completed, go to the menu System->Assign Base Addresses
  25. Save the design.
  26. Go to the menu Generate-> Generate HDL, and generate the Qsys system

  27. For this design, while the Qsys system is generated, we will get this message

    It is important to know that after the synthesize you must execute this tcl file, so that you don’t get any timing issues with the RAM. However, if you are using other kind of RAM controller you might have to avoid this point.

  28. Once the generation is complete, you get the following message, you have to press close, and close the Qsys.

  29. Now go back to Quartus and press on the tap “Files” on the window “Project navigator”, then right click on the folder files, press “add/remove files in project”

    and add the file “basic_nios.qip” to the project

  30. Now create a folder on the project directory, and name it “modules”

  31. Download the following Verilog files and paste them within the folder “Modules”

  32. Now add them to the Quartus project

  33. Now open the file “Line_Buffer”, and make sure that the line 91 looks like this:
  34. Open the file “CCD_Capture”, and make sure that the line 81 looks like this:
  35. Now perform double click on the Top module of the hierarchy  “D5M_Example”, and instantiate the added modules.

    //  REG/WIRE declarations
    wire reset_n;
    reg		[11:0]	rCCD_DATA;
    reg				rCCD_LVAL;
    reg				rCCD_FVAL;
    wire	[11:0]	mCCD_DATA;
    wire			mCCD_DVAL;
    wire			mCCD_DVAL_d;
    wire	[15:0]	X_Cont;
    wire	[15:0]	Y_Cont;
    wire	[31:0]	Frame_Cont;
    wire			DLY_RST_0;
    wire			DLY_RST_1;
    wire			DLY_RST_2;
    wire			DLY_RST_3;
    wire			DLY_RST_4;
    wire	[11:0]	sCCD_R;
    wire	[11:0]	sCCD_G;
    wire	[11:0]	sCCD_B;
    wire			sCCD_DVAL;
    wire auto_start;
    //  Structural coding
    assign reset_n = 1'b1;
    assign D5M_RESET_N=DLY_RST_1;
    assign D5M_TRIGGER=1'b1;
     assign	LEDG		=	Y_Cont;
    //auto start when power on
    assign auto_start = ((KEY[0])&&(DLY_RST_3)&&(!DLY_RST_4))? 1'b1:1'b0;
    //D5M read 
    always@(posedge D5M_PIXCLK)
    	rCCD_DATA	<=	D5M_D;
    	rCCD_LVAL	<=	D5M_LVAL;
    	rCCD_FVAL	<=	D5M_FVAL;
    //Reset module
    Reset_Delay			u2	(	.iCLK(CLOCK_50_B5B),
    //D5M image capture
    CCD_Capture			u3	(	
    //Bayer to RGB
    RAW2RGB				u4	(	
    // HDMI Config - HDMI I2C
    I2C_HDMI_Config u27 ( 
  36. Now it is time to instantiate the Qsys system on the top module, for that you have to go the Project Navigator and select “Files”, and then expand the file “basic_nios.qip”, select the file “basic_nios.v”, perform right click, and press on “Create Verilog Instantiation…”

  37. After this, a new file should appear on the project path, with the name of “basic_nios_inst.v”, open it and copy the instantiation.

  38. Once copied, you must paste it on the top Verilog module, below the already instantiated modules() as follows.

    basic_nios basic_nios_inst
    	//system clocks and reset
    	.clk_clk(CLOCK_50_B6A) ,	// input  clk_clk_sig
    	.reset_reset_n(DLY_RST_2), 	// input  reset_reset_n_sig
       .clk_0_clk                  (CLOCK_50_B6A),                                                  //                                clk_0.clk
       .reset_0_reset_n            (DLY_RST_2),                                            //                              reset_0.reset_n
       .clk_1_clk                  (CLOCK_50_B7A),                                                  //                                clk_1.clk
       .reset_1_reset_n            (DLY_RST_2),                                            //                              reset_1.reset_n
       .video_clocks_d5m_clk_clk   (D5M_XCLKIN),                                   //                 video_clocks_d5m_clk.clk
       .video_clocks_hdmi_clk_clk  (HDMI_TX_CLK) ,  
    	//LPDDR2 SDRAM Interface
    	.memory_mem_ca    (DDR2LP_CA),    // memory.mem_ca
       .memory_mem_ck    (DDR2LP_CK_p),    //       .mem_ck
       .memory_mem_ck_n  (DDR2LP_CK_n),  //       .mem_ck_n
       .memory_mem_cke   (DDR2LP_CKE),   //       .mem_cke
       .memory_mem_cs_n  (DDR2LP_CS_n),  //       .mem_cs_n
       .memory_mem_dm    (DDR2LP_DM),    //       .mem_dm
       .memory_mem_dq    (DDR2LP_DQ),    //       .mem_dq
       .memory_mem_dqs   (DDR2LP_DQS_p),   //       .mem_dqs
       .memory_mem_dqs_n (DDR2LP_DQS_n), //       .mem_dqs_n
       .oct_rzqin        (DDR2LP_OCT_RZQ),         //    oct.rzqin
       .mem_if_lpddr2_emif_0_pll_ref_clk_clk          (CLOCK_50_B5B),
       .mem_if_lpddr2_emif_0_status_local_init_done   (lpddr2_local_init_done),   // mem_if_lpddr2_emif_status.local_init_done
       .mem_if_lpddr2_emif_0_status_local_cal_success (lpddr2_local_cal_success), //                          .local_cal_success
       .mem_if_lpddr2_emif_0_status_local_cal_fail    (lpddr2_local_cal_fail),    //                          .local_cal_fail
    	 //camera input
       .camera_alt_vip_cti_0_clocked_video_vid_clk                   (D5M_PIXCLK),
       .camera_alt_vip_cti_0_clocked_video_vid_data                  ({sCCD_R[11:4],sCCD_G[11:4],sCCD_B[11:4]}),
       .camera_alt_vip_cti_0_clocked_video_vid_datavalid             (rCCD_LVAL),
       .camera_alt_vip_cti_0_clocked_video_vid_f                     (1'b0),
       .camera_alt_vip_cti_0_clocked_video_vid_h_sync                (rCCD_LVAL),
       .camera_alt_vip_cti_0_clocked_video_vid_locked                (DLY_RST_3),
       .camera_alt_vip_cti_0_clocked_video_vid_v_sync                (rCCD_FVAL),
    	//camera config
    	.camera_d5m_config_external_interface_SDAT (D5M_SDATA) ,	// inout  d5m_camera_config_external_interface_SDAT_sig
    	.camera_d5m_config_external_interface_SCLK (D5M_SCLK) ,	// output  d5m_camera_config_external_interface_SCLK_sig
    	//Video output
    	.alt_vip_itc_0_clocked_video_vid_clk		(HDMI_TX_CLK) ,	// input  alt_vip_itc_0_clocked_video_vid_clk_sig
    	.alt_vip_itc_0_clocked_video_vid_data		(HDMI_TX_D[23:0]) ,	// output [23:0] alt_vip_itc_0_clocked_video_vid_data_sig
    	.alt_vip_itc_0_clocked_video_vid_datavalid(HDMI_TX_DE) ,	// output  alt_vip_itc_0_clocked_video_vid_datavalid_sig
    	.alt_vip_itc_0_clocked_video_vid_v_sync	(HDMI_TX_VS) ,	// output  alt_vip_itc_0_clocked_video_vid_v_sync_sig
    	.alt_vip_itc_0_clocked_video_vid_h_sync	(HDMI_TX_HS) 	// output  alt_vip_itc_0_clocked_video_vid_h_sync_sig

  39. Now it is time to Synthesize, for that perform double click on window “Tasks”, specifically on the section “Analysis and Synthesis”.

  40. Once the synthesis is finished, and recalling the point 23 in which the Qsys Console gave us a message that said “Run the tcl script after synthesis and before fitting”, we need to go to the menu “Tools->Tcl Scripts…”, and look for the tcl file we want to run, and then just press on “run”.

  41. The after running the TCL file, we are free to close the TCL scripts window, and then double click on “Assembler(Generate Programing files)” on the window “Tasks”

  42. Finally we have finished the Hardware part, you can now download the <>.sof file to see the camera working on the HDMI, and then you can continue with the software part in which we are going to configure the camera exposure through the Nios II using the “Audio and Video Config” module.
  43. In addition, you can download the Hardware version from here, only as a guidance in case you have errors.

    The Hardware Solution you can see it running on the following video:






  1. Making sure you have completed the hardware section, and that your system works as you could see on the youtube video, it is time to write C code for the Nios II to change the Camera parameters such as Exposure, Camera color gains, among others.
    For this purpose it is important to Download the Datasheet of the camera that you can find on the following link.
  2. As we are using the “Audio and Video Config” Core, it would be useful to download the datasheet of this core as well. However, we are going to pore through the most important parts of the camera datasheet for this manual.
  3. Open the camera datasheet, and go to the page 10, and take a look at the table 1.7, and 1.8.
    these tables are divided into different columns, and some of them have below the title a number like this , those number are addresses that have to be set to the respective values to achieve certain resolutions and frame rates.
  4. In our particular case, the resolution at we are using the D5M camera is 1920x1080 which you can find on the table 1.8.
  5. By using the “Audio and Video Config” we can modify these registers to configure the camera as we whish.
    *Note: there are other registers that you can find in detail in the page 11, chapter 2.
  6. For this example, we are going to write code to modify the different registers of the D5M camera.
  7. Open the “Nios II XX.X Software Build Tools for Eclipse”, and select the workspace “fpgas”.
  8. Now on the menu go to “File->Nios II application and BSP from template”.
  9. Select the SOPC (basic_nios. sopcinfo) File that you can find in the hardware project directory and contains the specifications for the Nios II. Then, name the Project as “Camera”, select the Project Template “Blank Project”, and press finish.

  10. Once you press finish, on the project explorer there should be two folders, one with the name “Camera”, and the other with the name “Camera_bsp”.
  11. Perform right click on the folder “Camera_bsp” and go to “Nios II->BSP Editor…”
  12. On settings go to common, and select “make“ and change the bsp optimization to -O2, then press Generate and exit.

  13. Now perform right click on the folder “Camera”, press on properties, then select “Nios II Application Properties”, and change the Optimization Level to 2.

  14. Now right click on the folder “Camera”, and create a new “Source file”, and name it “main_camera.c”.

  15. Now copy the following code on the created file “main_camera.c”, save it, and analyze the code, comparing it with the D5M Datasheet, and the mentioned registers on the table 1.7, and 1.8.
    Written by Holguer Andres Becerra
    for more go to
    #include <stdio.h>
    #include <stdlib.h>
    #include <system.h>
    #include "io.h"
    #include "altera_up_avalon_audio_and_video_config.h"
    #define D5M_COLUMN_SIZE	1919
    #define D5M_ROW_SIZE	1079
    #define D5M_COLUMN_BIN	0x0000
    #define D5M_ROW_BIN		0x0000
    #define EXPOSURE_LIMIT 1079
    //Registers Addresses
    unsigned char address[25] = { 0x00, 0x20, 0x09, 0x05, 0x06, 0x0A, 0x2B, 0x2C,
    		0x2D, 0x2E, 0x10, 0x11, 0x12, 0x10, 0x98, 0xA0, 0xA1, 0xA2, 0x01, 0x02,
    		0x03, 0x04, 0x22, 0x23, 0x49 };
    //Registers Values
    unsigned short data[25] = { 0x0000, 0xc000, EXPOSURE_LIMIT, 0x0000, 0x0019, 0x8000,
    		0x000b, 0x000f, 0x000f, 0x000b, 0x0051, 0x1807, 0x0002, 0x0053, 0x0000,
    		0x0000, 0x0000, 0x0FFF, 0x0036, 0x0010, D5M_ROW_SIZE, D5M_COLUMN_SIZE, D5M_ROW_BIN, D5M_COLUMN_BIN,
    		0x01A8 };
    int i;
    void init_d5m_camera(alt_up_av_config_dev * d5m) {
    	d5m->type = TRDB_D5M_CONFIG;
    	d5m->type = TRDB_D5M_CONFIG;
    	for (i = 0; i < 25; i++) {
    		alt_up_av_config_write_D5M_cfg_register(d5m, address[i], data[i]);
    void set_d5m_exposure(alt_up_av_config_dev * d5m,unsigned short exposure){
    	alt_up_av_config_write_D5M_cfg_register(d5m, 0x09, exposure);
    int main() {
    	alt_up_av_config_dev * d5m= alt_up_av_config_open_dev("/dev/camera_d5m_config");
    	unsigned short exposure = 0;
    	while (1) {
    		exposure += 100;
    		printf("exposure %d\n", exposure);
    	return 0;
  16. Right click on the folder “Camera”, and build project.
  17. If everything was successful, you will have inside the folder “Camera” a new file called “Camera.elf”
  18. Go to Quartus, open the programmer and download the .sof file to the FPGA.
  19. Go back to Eclipse, right click on the folder “Camera”, and press “Run As-> Nios II Hardware”

  20. You can see the results on the following youtube video.

Download this file (basic_nios.qsys)basic_nios[Qsys Template]31 kB
Download this file ( hw[Solution]949 kB
Download this file ( builder Template[System Builder Template]7 kB
Download this file (plls_camera_hdmi.qsys)PLLs[PLLs for camera and HDMI]26 kB
Download this file (TRDB-D5M_Hardware specification_V0.2.pdf)TRDB-D5M_Hardware specification_V0.2.pdf[D5M Datasheet]921 kB
Download this file (video_d5m_input5.qsys)Camera Input Subsystem[Camera Qsys Input Subsystem]9 kB
Powered by OrdaSoft!
  Written By Peter Gomez Este contenido esta orientado a los programadores que tienen problema con la conectividad("SGC PmmC") de su pantalla uOled-128-g1/g2…
Written by Sherneyko Plata Rangel   Pynq-z2: Hello world   In this tutorial we will implement a simple test of the inputs/outputs available on…
Objetivos Requerimientos Procedimiento Descripción de Hardware. Qsys. Nios II. UCOS II. Secuencia de Sprite. Sintesis de Audio. Descargas Glosario Otros Resultados. Ejemplo de Sprites.     Objetivos: Diseñar una plantilla general para el diseño de…
Written by Holguer A. Becerra           Requerimientos: DE0-NANO USB-UART(solo para parte 3) Python 2.7 ó superior.   Objetivos: Dar una introducción a los conceptos de Multitasking, Scheduling y…
Written by Holguer A. Becerra             Based on Gregory Estrade's Work.   I have ported the PC Engine System on the DE0-NANO back in 2014, why…
      Arduino tools are generally nice tools for quick prototyping and improvized projects, and the Seeeduino Xiao…
Written by: Holguer A Becerra         En esta practica vamos a construir nuestro primer juego retro  usando un sincronizador de Video VGA…
Written by: Andrea Paola Pabón Ortega & Daniel Enrique Mejia Rueda Revision by: Ing Holguer A. Becerra   DESCRIPCIÓN DEL PROYECTO: El  RTAWD DE0NANO…
  Written by Holguer Andres   Requires: DE0-NANO. 4.3 Inch 480x272 Screen.( WQVGA ) ?️       Parte HW: Descargue la siguiente plantilla( y descomprimala en una ruta sin espacios y…
Escrito por Guillermo Acevedo   Diseño   En esta practica desarrollaremos un filtro FIR en hardware, para este caso en especifico, realizaremos un filtro…
 Written By Juan David Delgado   FILTRO FIR (FILTRO DE RESPUESTA FINITA AL IMPULSO)     Son conocidos como filtros digitales no recursivos debido a…
XISCKER: Reduced and Complex Instruction Set Computing Key Educational Resources A Configurable Platform for the Emulation of Different Computer Architectures An introduction to Computer Architectures through digital design description for FPGA devices   Computer Architecture embraces all three…
Escrito por: Alix Angarita   En el manual a continuación se explica un método de debug adicional que es muy interesante debido a…
By: Fabio Hernández   INTRODUCCIÓN:   El presente documento pretende mostrar la manera de generar software para una imagen de Linux implementada en la…
Summary Written by Fabio Hernandez   HARD PROCESSOR SYSTEM (HPS)     ------------------------------------------------------------------------------------------------------------------------------------------------   Introducción   Tenemos  2 nuclos de procesamiento ARM cortex-A9, cada uno son su propio cache  se…
Escrito por Jesus Lopez         INTRODUCCIÓN   El acceso directo a memoria (DMA, del inglés direct memory access) permite a cierto tipo de componentes de una computadora acceder a…
    Written by  Sebastian Baquero       Objetivos  Introducción a los conceptos de Multitasking, Scheduling y Context Switching.  Ampliación de los conceptos a cerca de el…