Purpose:
After completing this lab, you will appreciate the power of the MPLAB® Harmony Graphics Composer (MHGC), by discovering the ease of adding display graphics functionality to the end application. You will also learn how to integrate the new functionality into the existing MPLAB Harmony project.
Overview:
In this lab, you will add to the previous MPLAB Harmony project (audio_player_lab4) a graphics display UI on the PIC32 Multimedia Expansion Board (MEB) II. The audio_player_lab5 application displays audio tracks from the SD card in a list box. You can navigate the list to select and play a track. It also provides a volume increase/decrease scroll bar and a mute on/off button. This lab will demonstrate the configuration and integration of additional modules to audio_player_lab4 using the MPLAB Harmony Configurator (MHC), thereby extending its functionality.
Note: The state machine for
APP_SDCARD_AUDIO_Tasks remains the same as explained in
audio_player_lab4.
The APP_DISPLAY_Tasks() does not implement an internal state machine. It implements a set of functions based on events in the audio application.
Lab Source Files and Solutions:
If you haven't already, download the lab source files and solutions for the SD card audio player labs.
This project has been verified to work with the following versions of software tools:
MPLAB X IDE v3.26, MPLAB XC32 Compiler v1.40, MPLAB Harmony v1.08
Because we regularly update our tools, occasionally you may discover an issue while using newer versions. If you suspect that to be the case, we recommend that you double check using the same versions that the project was tested with.
Archived versions of our tools can be found on the following Microchip websites:
MPLAB Harmony (See the Archived Downloads tab.)
MPLAB X IDE and XC32 Compiler (See the Downloads Archive tab.)
Note that multiple versions of all these tools can co-exist on the same computer.
Procedure:
This lab builds off the work you performed in the previous lab. If you did not perform SD card Audio Player Lab4, please start this lab using the Lab4 solution project. Open that project (found under the lab's firmware folder) and verify it works as expected before continuing with this lab.
All steps must be completed before you will be ready to build, download, and run the application.
Lab Index
Step 1: Copy source files and rename project for Lab5
Create a new folder audio_player_lab5 under apps/training/middleware/dev/audio_player.
At this point, we suggest you close any open projects from previous labs. This will avoid confusion since you will be opening the project with the same name as the previous lab before renaming it for this lab.
Copy the firmware folder from apps/training/middleware/dev/audio_player/audio_player_lab4 to the newly created folder (audio_player_lab5).
Rename the project from audio_player_lab4 to audio_player_lab5, because by the end of this session the lab will have audio_player_lab5 funcitonality. Open audio_player_lab4.
Under Projects, right click on audio_player_lab4 and select Rename….
In the pop-up window, rename the lab from audio_player_lab4 to audio_player_lab5.
The MPLAB X IDE renames the project. It closes the old project audio_player_lab4 and opens the renamed project audio_player_lab5.
Verify that the project builds properly after renaming. Click the
Clean and Build icon:
In the MPLAB X IDE Projects pane, right click on audio_player_lab5 and select Set as Main Project.
Open MHC. In MPLAB X IDE click on Tools > Embedded > MPLAB Harmony Configurator.
Step 2: With MHGC under MHC, Design Display GUI
Step 2.1: Rename the Screen Name
Open MHGC. To do this, expand the Harmony Framework Configuration > Graphics Library > Harmony Graphics Library selection tree, and check the Use Graphics Library? box. Click the Execute button beside Create a Design With MPLAB Harmony Graphics Composer.
Change the screen name from "Screen1" to "MchpMainScreen". Find the Graphics Composer Management tab in the bottom left pane and double-click on the Screen1 name to change it.
You are not changing any of the properties of the screen, so keep the screen properties shown in the Graphics Composer Properties tab in the top-right pane.
Step 2.2: Import Images and Fonts (Assets)
Import the following images and fonts. These will be used for the display objects created in the "Add Primitive Objects" step below.
- Images:
- The Microchip logo will be applied to the LogoImage primitive object.
- Two images will be applied to the VolButton widget. One for mute off (play) and one for mute on.
- System Font:
- Arial font, 14 point for ASCII numbers 32 to 127. This will be used for multiple schemes in the design.
- Arial font, 18 point for ASCII number 94. This will be used for the ScrollUpScheme, and applied to the ScrollUpButton widget.
The text displayed in the ScrollDownButton is a lowercase "v". The text displayed in the ScrollUpButton is a "caret" or "^" (ASCII 94).
Copy the Icons folder from this folder:
apps/training/middleware/audio_player/audio_player_lab5/dev_files
to this one:
apps/training/middleware/dev/audio_player/audio_player_lab5.
In the bottom left pane, in the Graphics Composer Management tab, click on the Assets sub tab. Import the following assets:
Import these images:
- MCHP_LOGO.bmp
- AudioMute16_2.bmp
- AudioPlay16_2.bmp
from this folder:
apps/training/middleware/dev/audio_player/audio_player_lab5/Icons
Import "System Font" Arial 14pt from 32 ASCII to 127 ASCII.
Click on Edit Import Ranges and select the required character range. This is an important feature that enables you to import only the characters you need, thereby limiting the amount of memory needed to store the font.
- Start Index indicates the starting ASCII value of the character.
- End Index indicates the last ASCII value of the character.
Alternatively, the range can be provided by Start Character and End Character.
Import "System Font" Arial 18pt for 94 ASCII.
Step 2.3: Create Schemes (assign standard colors and fonts)
Create schemes to standardize the look and feel (fonts, colors, styles) of the user interface. These will be used for the display widgets created in the "Add Widget Objects" step below.
- Images:
- GroupScheme
- Applied to the AudioModeGroup widget
- RadioScheme
- Applied to the application mode buttons RadioButtonPlayer and RadioButtonReader widgets
- TrackListScheme
- Applied to the TrackListBox, ScrollDownButton, and VolSlider widgets
- ScrollUpScheme
- Applied to the ScrollUpButton widget
- VolumeButtonScheme
- Applied to the VolButton widget
In the bottom left pane, in the Graphics Composer Management tab, click on the Schemes sub tab. A default scheme is present.
Create following custom schemes by clicking the Create button.
Step 2.4: Add Primitive Objects (rectangle, image, text)
Design the following primitive objects:
- BgRectangle: Rectangle with a gradient shade at the top of the screen
- LogoImage: Image of Microchip logo at the top of the screen
- AppNameText: Text to display the application name
BgRectangle:
In the Graphics Composer Tool Box (top left pane), under the "Primitives" group, left-click and drag Rectangle onto the Graphic Composer Screen.
In the Graphics Composer Properties tab (top right pane), observe the properties of this Rectangle.
Modify these properties as shown below:
The screen will now look like this:
LogoImage:
In the Graphics Composer Tool Box (top left pane), under the "Primitives" group, left-click and drag Image onto the Graphic Composer Screen.
Modify the properties of the image through the Graphics Composer Properties as follows:
The screen will now look like this:
AppNameText:
In the Graphics Composer Tool Box, under the "Primitives" group, left-click and drag Text onto the Graphic Composer Screen.
Modify the properties of the text through the Graphics Composer Properties as follows:
The screen will now look like this:
Step 2.5: Add Widget Objects (buttons, boxes, slider)
Design the following widgets:
AudioModeGroup:
In the Graphics Composer Tool Box, under the "Widgets" group, left-click and drag Group Box onto the Graphic Composer Screen.
Modify the properties of the group box through the Graphics Composer Properties as follows:
Notice the scheme GroupScheme is applied to the AudioModeGroup group box.
The screen will now look like this:
RadioButtonPlayer:
In the Graphics Composer Tool Box, left-click and drag Radio button onto the Graphic Composer Screen.
Modify the properties of the radio button through the Graphics Composer Properties as follows:
Note that the Check option is selected. This selects the SD card "Player" mode by default.
The screen will now look like this:
Notice that the radio button event Checked is checked. Click on the browse button (beside the Checked box) to add/edit action to the radio button event. This opens an Event Editor window. Click on the Add button to add an action to the Clicked radio button event.
In the Action window, select Type of action as Custom. Rename the action from "Action" to "CustomAction". Copy the following code into the Custom Action Definition window.
This code re-enables the SD card audio player related user interface (UI) elements, and puts the application in "Player" mode.
RadioButtonReader:
In the Graphics Composer Tool Box, left-click and drag Radio button onto the Graphic Composer Screen.
Modify the properties of the radio button through the Graphics Composer Properties as follows:
The screen will now look like this:
Notice that the radio button event Checked is checked. Click on the browse button (beside the Checked box) to add/edit action to the radio button event. This opens an Event Editor window. Click on the Add button to add an action to the Clicked radio button event.
In the Action window, select Type of action as Custom. Rename the action from "Action" to "CustomAction". Copy the following code into the Custom Action Definition window.
This code disables the SD card audio player related UI elements, and puts the application in USB SD card "Reader" mode.
TrackListBox:
In the Graphics Composer Tool Box, left-click and drag List Box onto the Graphic Composer Screen.
Modify the properties of the list box through the Graphics Composer Properties as follows:
Notice that the list box event Touched is checked. Click on the browse button to add/edit action to the list box event. This opens an Event Editor window. Click on the Add button to add an action to the Touched list box event.
In the Action window, select Type of action as Custom. Rename the action to CustomAction, and copy the following code into the Custom Action Definition window.
This code switches to the selected random track so that the selected random track is played.
The screen will now look like this:
ScrollUpButton:
In the Graphics Composer Tool Box, left-click and drag Button onto the Graphic Composer Screen.
Modify the properties of the button through the Graphics Composer Properties as follows:
Notice that the button event Released is checked. Click on the browse button to add a custom action to the button event.
In the Action window, select Type of action as Custom. Rename the action to CustomAction. Copy the following code into the Custom Action Definition window.
This code sets the focus in the list box to one item above (previous item) the current item in focus, when a button release event is triggered.
The screen will now look like this:
ScrollDownButton:
In the Graphics Composer Tool Box, left-click and drag Button onto the Graphic Composer Screen.
Modify the properties of the button through the Graphics Composer Properties as follows:
Notice that the button event Released is checked. Click on the browse button to add a custom action to the button event.
In the Action window, select Type of action as Custom. Rename the action to CustomAction. Copy the following code into the Custom Action Definition window.
This code sets the focus in the list box to one item below (next item) the current item in focus, when the button release event is triggered.
The screen will now look like this:
VolSlider:
In the Graphics Composer Tool Box, left-click and drag Slider onto the Graphic Composer Screen.
Modify the properties of the slider through the Graphics Composer Properties as follows:
Notice that the slider events Incremented and Decremented are checked. Click on the browse button to add a custom action to the slider events.
In the Action window, select Type of action as Custom. Rename the action to CustomAction. Copy the following code into the Custom Action Definition window for both events.
This code increments/decrements the audio volume based on the position of the volume slider.
The screen will now look like this:
VolButton:
In the Graphics Composer Tool Box, left-click and drag Button onto the Graphic Composer Screen.
Modify the properties of the button through the Graphics Composer Properties as follows:
Notice that the button events Pressed and Released are checked. Also, notice that the button has enabled the Toggle option. This option allows the button to be used as an ON/OFF button. Also notice the default state is Pressed. The Pressed state corresponds to Mute Off, and the Released state corresponds to Mute On.
Click the browse button to add a custom action to the button events.
In the Action window, select Type of action as Custom. Rename the action to CustomAction. Copy the following code into the Custom Action Definition window for the Pressed event.
Copy the following code for the Released event.
The screen will now look like this:
MessageTextBox:
In the Graphics Composer Tool Box, left-click and drag Static Text onto the Graphic Composer Screen.
Modify the properties of the static text through the Graphics Composer Properties as follows:
Since the Visible property of the Static Text is unchecked, the static text will not be visible on the screen. Check the Visible property to verify the text, and then un-check it again.
With the Visible property checked, the screen will look like this:
Click on MHC's Options tab to verify the Graphics Library has been properly configured as shown in the following image.
Step 3: With MHC, Configure Touchscreen, and I2C driver
Step 3.1: Enable Touchscreen Control using I2C
Enable the touch control feature of the graphics display on the MEB II board.
In MHC's Options tab, expand the Harmony Framework Configuration > Drivers > Graphics Displays > Use Graphics Display? selection tree. Notice the display has been correctly selected for you (MEB II uses a 4.3 inch, WQVGA display by NewHaven).
Expand Graphics Displays > Display Settings. You will keep all settings as is.
The Display Settings have values specific to the display (4.3 inch, WQVGA display by NewHaven). These include horizontal and vertical timing parameters and others. For details, refer to the display data sheet.
Configure the display Touch driver (MTCH6301) to use I²C driver instance 1.
Note I²C Driver Instance 1 hasn't been created yet. You will do this a couple of steps from now.
Expand the Harmony Framework Configuration > Drivers > Touch > MTCH6301 > Use the MTCH6301 Driver? selection tree.
Keep all options the same, except for the "I2C driver module index". Change this to use "DRV_I2C_INDEX_1" (I²C Driver Instance 1).
Verify/Set the I/O pins used by the Touch driver using the Graphical Pin Manager.
The touch driver uses PIC32 external interrupts source 1 for touch events. Select MHC's Pin Diagram tab, and in the lower pane, select MHC's Pin Table tab.
Map the External Interrupt 1 signal to pin RE8 as shown in this image. (For the schematic, refer to the MEB II User’s Guide).
Add a new client and driver instance for the existing I²C driver.
In audio_player_lab1, you created a Dynamic I²C driver and created a client for this driver. You also configured an I²C driver instance (I²C Driver Instance 0) for this client. This client and instance are used by the AK4953 audio CODEC.
In this step, you will add another client and instance (I²C Driver Instance 1) for this driver for the display Touch driver (MTCH6301).
Expand the Harmony Framework Configuration > Drivers > I2C > Use I2C Driver? selection tree.
Modify Number of I2C Driver Clients and Number of I2C Driver Instances to 2.
You configured I2C Driver Instance 0 in audio_player_lab1. Just verify Use Bit Bang I2C Implementation is still unchecked.
Expand I2C Driver Instance 1, and make sure Use Bit Bang I2C Implementation? is unchecked. Since you are using the PIC32 I2C peripherals, you do not want to use the bit banged driver implementation.
Retain I2C_ID_1 as the I2C Module ID.
I2C_ID_1 refers to the PIC32 I²C1 peripheral (using pins SCL1 & SDA1). These pins are connected to the display touch controller on the MEB II board.
Retain default value DRV_I2C_MODE_MASTER as Operation Mode. The PIC32 I²C module 1 will be the master, and the Touch MTCH6301 interface will be the Slave.
Change value for Master Interrupt Priority and Master Interrupt Sub-priority as INT_PRIORITY_LEVEL4 and INT_SUBPRIORITY_LEVEL0 respectively. Lower priority for touch events is OK as this will not be occurring very frequently.
Change value for Error Interrupt Priority and Error Interrupt Sub-priority as INT_PRIORITY_LEVEL4 and INT_SUBPRIORITY_LEVEL0 respectively.
The Baud Rate generator clock is configured for 100 MHz. This is derived from the Peripheral bus 2 clock frequency generated from the 198 MHz system clock.
Retain the default value of 50 kHz for I2C CLOCK FREQUENCY.
Retain the Slew Rate Control as unchecked. This function enables the I2C module to use high-frequency signaling, allowing it to use the 400 kHz and 1 MHz signaling rates.
Retain the default value of SYS_MODULE_POWER_RUN_FULL for Power State. You will want to run the I²C module in normal power mode.
Step 3.2: Verify I/O Pins used by I2C module
Click on MHC's Pin Diagram tab, and in MHC's lower pane, click on the Pin table tab.
The I2C1 has dedicated pins for SCL1 and SDA1 signals. These pins are hardcoded and configured by the I²C driver.
Step 3.3: Enable Touch System Service
The Harmony touch display driver is used by the Harmony Touch System service. Expand the Harmony Framework Configuration > System Services > Touch selection tree, and check the Use Touch System Service box.
Save the MHC configuration by clicking the save button.
Step 4: Generate Harmony code
Generate and save the configuration as shown below :
Resolve code generation merge conflicts.
When you click the Generate button, MHC will modify and generate source files based on the options selected in MHC. If MHC generates code that will change a pre-existing file, it will show you the code it generated, and allow you to confirm if you want to add it to your source code. The following screen shots will guide you on which changes need to be merged (i.e., accepted).
Click the arrow (highlighted in green circle) to copy and merge from the Generated Code to the Current file.
Click the Next button (highlighted in blue) to move to next set of merge conflict.
Note that the following definition needs to be retained in the Current File:
Examine what was done after “Generating Code”.
The app folder contains files related to your specific application. In the following steps, you will add your application code to the existing app.c and app.h files. You will also add other application specific source files to the project.
The framework folder under the app folder (app/system_config/pic32mz_ef_sk_meb2/framework) contains customized Harmony Framework files. These files have been generated by MHC in response to your specific MHC selections.
Also note MHC has generated three custom graphics source files for you, based on the selections you made in the previous two steps. It has placed these new source files in the Source Files/app/system_config/pic32mz_ef_sk_meb2 folder.
Harmony groups all source files supporting a specific hardware platform into their own system_config folder (named pic32mz_ef_sk_meb2 in this case). Doing this enables one MPLAB X IDE project to support multiple hardware platforms by using its project configurations feature. "Project configurations" includes or excludes hardware specific configuration folders based on the target hardware you select for a specific build.
The framework folder under the Source Files folder contains standard Harmony Framework files that have been added to your project by MHC (based on your MHC selections). To be clear, unlike the framework files found under the app folder, these files have not been modified by MHC. They are some of the standard Harmony Framework files included in the Harmony Framework download.
You can see MHC has added a folder named gfx under the Source Files/framework/driver folder. It has also added another folder named gfx under the Source Files/framework folder for the Graphics Library files.
If you were to build the project now, the build would fail (linker error: undefined reference to APP_DISPLAY_VolumeMuteChange). The event handler of the display object calls this function. You will add the application display code to implement this API in the next step.
Step 5: Include application specific source files, add required code and build the project
The app_button_press_task.c and app_button_press_task.h files are no longer needed as it's functionality will be replaced by the radio buttons on the display. Right click on these files and select Remove From Project.
Copy source files into your project's Source Files folder:
- app_display_task.c
- app_display_task.h
Copy them from this folder:
apps/training/middleware/audio_player/audio_player_lab5/dev_files
to this one:
apps/training/middleware/dev/audio_player/audio_player_lab5/firmware/src.
Add the copied source files to your project.
- Add app_display_task.c to the Source Files/app folder (in the MPLAB X IDE Projects pane) by right clicking and selecting Add Existing Items…
- Add app_display_task.h to the Header Files/app folder by right clicking and selecting Add Existing Items…
The files under the project should look like this:
As you did in the previous labs, modify the app.c and app.h files from audio_player_lab4 to replace the Button task related API with the Display task related API in APP_Initialize and APP_Tasks functions.
Notice how easy it was to integrate the display code to the existing audio_player_lab4 code. This is primarily due to Harmony's state based implementation (details of implementation follow).
Now that you have added graphics to this lab, the SD card audio task needs to notify the display task whenever the tracks list is updated. Open the app_sdcard_audio_task.c file and add the following function call under the APP_SDCARD_AUDIO_CARD_STATE_INIT state.
Finally, you need to add application specific definitions in the system_config.h file. Add the #define statement shown below.
Save all the files before closing.
Now, you are ready to build the code. Click the
Clean and Build icon and verify that the project builds successfully.
Step 6: Review the Application code
Application File: app_sdcard_audio_task.h
This file remains the same as in audio_player_lab4. No changes are needed. Refer Review the Application code section in Lab4
Application File: app_display _task.h
Open file app_ display _task.h (in the MPLAB X IDE Projects pane under the Header Files > app folder). This file defines application enums, data, and APIs.
The volume mute enumerator helps manage the current audio activity, and acts (on a button press) to mute and un-mute, or control volume increase and decrease.
The app mode enumerator helps manage the mode selected by the user on the UI.
The Display data structure holds the volume/mute information, the ID of the current track being played, and a handle to system timer for implementing de-bouncing for the volume slider.
Application File: app_sdcard_audio_task.c
This file remains the same as in audio_player_lab4, except for a function call that is added to notify the display task of any changes to the tracks list. No other changes are needed. Review the Application code section in Lab4
Application File: app_display_task.c
Open file app_display_task.c (in the MPLAB X IDE Projects pane under the Source Files > app folder). This file contains the display task initialization and task functions.
The function APP_DISPAY_Initialize sets up the default value for the volume/mute variable as not changed.
The function APP_DISPLAY_Tasks performs the following tasks:
Once the display screen has been drawn by the gfx library state machine, it initializes the listbox object with tracks from the SD card, whenever the tracks list is updated by the SD card audio task. It also sets the default volume level in the volume slider.
It handles changes to the application mode. When the user switches from the SD card player mode to USB SD card reader mode, it suspends the streaming, attaches the USB device and then switches the application mode to reader mode. When the user switches back to SD card player mode, it re-initializes the SD card audio task, detaches the USB device and then switches the application mode to player mode.
It handles the volume and mute control. The user can increase or decrease the volume through the volume slider. For mute control, the volume slider is disabled or enabled based on whether the volume is mute or un-mute. Also, the audio CODEC is updated with the new volume level. Note that the volume slider has de-bounce logic implemented by registering a call-back with the system timer service. This avoids having multiple I²C transactions to the audio CODEC for volume changes.
It updates the list box object with the current audio track being played. The audio files are played one after the other from the SD card, and hence, when the current audio track changes, the list box object’s focus is updated to reflect the current (changed) track being played.
The function _APP_DISPLAY_HandleMuteOn handles the volume mute event by setting the level of the volume slider object to zero and then disabling the volume slider object.
The function _APP_DISPLAY_HandleMuteOff reverses the action by setting the level of the volume slider object to its previous position and then enabling the volume slider object.
The function _APP_DISPLAY_HandleVolumeIncreaseDecrease scales the level of volume slider object to the volume level of audio CODEC.
Step 7: Debug your Application
Congratulations! You are ready to DEBUG
your fifth application!
Before you program and run our application, make sure that the micro SD card has at least one audio files saved on it.
Make sure that the micro SD card is inserted into the micro SD card slot (J8) on the MEB II.
Before you start the debugger, you may want to set a breakpoint in the application file in app_display_task.c to verify that important stages of the application are executed successfully.
Put a breakpoint in APP_DISPLAY_Tasks which will indicate that the graphics library has finished drawing the current screen, and that the application is now ready to populate the list box object with the tracks list.
To gain a better understanding of graphics objects event handling, and how the custom actions are executed, you may want to put breakpoints in the GFX_HGC_MsgButtons function (gfx_hgc_definitions.c file).
Note that events retlated to the "volume slider" and "track list box" objects are handled by functions GFX_HGC_MsgScrollBars and GFX_HGC_MsgListBoxes respectively.
Debug your application! Click the “Debug Main Project”
icon.
Remove all breakpoints and press key "F5" to allow the application to run.
You should see the display now. Notice that the display looks exactly the same as MHGC's simulated display.
Experience the different features of the user interface.
- Change the application mode from 'player' to 'reader' by selecting the radio buttons
- Press the scroll up /scroll down buttons to navigate through the track list
- Increase/decrease the volume by changing the volume slider position
- Mute/Un-mute the volume by pressing on the volume mute button
Connect a headphone to the HP Out connector on the MEB II and listen to your favorite music. Notice the tracks list will scroll as the tracks get played out one after the other.
Results
If everything worked well, you should see a graphical user interface on the MEB II display, populated with a list of tracks read from the SD card along with volume and mute control. The user interface should respond to touch events and you should be able to select random tracks for playing, increase or decrease volume level and mute the audio output. You should be able to put the application either into audio player mode or SD card reader mode by selecting the appropriate radio button on the UI. You might not get the intended results if the Graphics Library was not configured correctly, an invalid Graphics Display was selected or if the Touch interface was not configured correctly.
Analysis
In this lab, you added graphics display support. You used the MHGC, embedded within MHC, to design and develop the graphical user interface. You added a display driver and touch driver from the list of driver implementations provided by MHC. You added custom actions to handle various UI events like touch, pressed, released etc. At the application layer, you added a new task (state machine) to handle graphics related activities like populating the display with the list of tracks read from the SD card, handling track change event, volume increase/decrease and mute events.
Conclusions
With this lab, you add graphics support to your application and extend it a step further. The lab demonstrates the use of the MHGC to design and develop UI screens. It shows you how to add and customize different properties of widgets, create schemes, add images and font libraries and how to handle events originating from the widgets. The lab shows how easily you can add complex graphics, display driver, and touch screen drivers to your project using MHC. Going forward, you can experiment with MHGC and try to create a UI of your choice. You may want to add multiple screens!
