Purpose:
After completing this lab, you will appreciate the power of the MPLAB® Harmony 3 Graphics Composer (MHGC), by discovering the ease of adding display graphics functionality to the end application. You will also learn how to integrate the new functionality into the existing MPLAB Harmony project.
Overview:
In this lab, you will add to the previous MPLAB Harmony project audio_player_lab4, a graphics display UI on the PIC32 Multimedia Expansion Board (MEB) II. The audio_player_lab5 application displays audio tracks from the SD card in a list box. You can navigate the list to select and play a track. It also provides a volume increase/decrease scroll bar and a mute on/off button. This lab will demonstrate the configuration and integration of additional modules to audio_player_lab4 using the MPLAB Harmony 3 Configurator (MHC), thereby extending its functionality.
Note: The state machine for
APP_SDCARD_AUDIO_Tasks remains the same as explained in
audio_player_lab4.
The APP_DISPLAY_Tasks() does not implement an internal state machine. It implements a set of functions based on events in the audio application.
Lab Source Files and Solutions:
If you haven't already, download the lab source files and solutions for the SD card audio player labs.
This project has been verified to work with the following versions of software tools:
MPLAB X IDE v5.25, MPLAB XC32 Compiler v2.30, MPLAB Harmony MHC v3.3.2, DEV_PACKS v3.5.0, CORE v3.5.0, CSP v3.5.0, BSP v3.5.0, AUDIO v3.4.0, USB v3.3.0, GFX v3.4.0, TOUCH v3.4.0
As the tools are regularly updated, there may be occasional issues while using newer versions. If that is the case, we recommend using the same version as specified in the project.
The archived versions of our tools can be found below:
MPLAB Harmony
MPLAB X IDE and XC32 Compiler
Note: that multiple versions of all these tools can co-exist on the same computer.
Procedure:
This lab builds off the work you performed in the previous lab. If you did not complete SD card Audio Player Lab 4, please start this lab using the Lab 4 solution project. Open that project (found under the lab's firmware folder) and verify it works as expected before continuing with this lab.
All steps must be completed before you will be ready to build, download, and run the application.
Lab Index
Step 1: Copy Source Files and Rename Project for Lab 5
Create a new folder audio_player_lab5 under apps/training/solutions/dev/audio_player.
At this point, we suggest you close any open projects from previous labs. This will avoid confusion since you will be opening the project with the same name as the previous lab before renaming it for this lab.
Copy the firmware folder from apps/training/solutions/dev/audio_player/audio_player_lab4 to the newly created folder (audio_player_lab5).
Rename the project from audio_player_lab4 to "audio_player_lab5", because by the end of this session the lab will have audio_player_lab5 funcitonality. Open audio_player_lab4.
Under Projects, right click on audio_player_lab4 and select Rename….
In the pop-up window, rename the lab from audio_player_lab4 to "audio_player_lab5".
The MPLAB X IDE renames the project. It closes the old project audio_player_lab4 and opens the renamed project audio_player_lab5.
Verify that the project builds properly after renaming. Click the
Clean and Build icon:
In the MPLAB X IDE Projects pane, right click on audio_player_lab5 and select Set as Main Project.
Open MHC. In MPLAB X IDE click on Tools > Embedded > MPLAB Harmony 3Configurator.
Step 2: Add Graphics and Touchscreen Components in MHC
Select the MHC Project Graph tab. Remember, if you ever lose the MHC Tool, you can find it here: Tools > Embedded > MPLAB Harmony 3 Configurator.
Add the Aria Graphics w/PDA TM4301B Display template into your project from the Available Components window.
Expand the tree Graphics > Templates and select Aria Graphics w/PDA TM4301B Display.
At this point, you will be asked to auto activate some components required by the system. Select Yes for all.
Your Project Graph should have added multiple components. Arrange the boxes to resemble the screen shown below.
Set the Display Interface to LCC.
Select the Aria Graphics w/PDA TM4301B Display component. This will display the current settings in the Configuration Options window. Set the Display Interface to LCC.
Revise the interface of the I2C Bit Bang component for the touch screen interface.
Right click on the yellow diamond in the I2C_BB component and select I2C (drv_i2c) from the list of consumers.
This will create a new instance for the I2C diver interface. See the simplified component diagram below:
Disconnect the current I2C driver interface from the MaxTouch Controller by right clicking on the DRV_I2C diamond and selecting Disconnect.
Reconnect the I2C diver interface by right clicking on the DRV_I2C diamond in Instance 1 interface. Then select MaxTouch Controller (gfx_maxtouch_controller) from the list of Consumers.
Your Project Graph should now resemble the screen shown below.
The graphic libraries require additional heap space. Click on the System component and expand the Configuration Options tree to display the Heap Size entry field. Set this to at least 32,768.
Click on Instance 1 of the I2C driver. In the Configuration Options, set Transfer Queue Size to 9.
Save the MHC configuration by clicking the save button.
Step 3: Build Display Screen Using MPLAB Harmony Graphics Composer (MHGC)
Step 3.1: Open MHGC and Setup the Initial Display Screen
Open the MPLAB Harmony Graphics Composer utility.
To do this, expand the MHC> Tools selection and click on Graphics Composer.
A Welcome Dialog screen will appear. Click Use design loaded from current MHC configuration or just close the screen to start.
The MPLAB Harmony Graphics Composer screen will be displayed as shown below.
Change the screen name.
In the Screens window, double click on the Name field and replace the name with "MchpMainScreen".
Step 3.2: Import Graphic Images
Copy the Icons folder from this folder:
apps/training/solutions/audio_player/audio_player_lab5/dev_files
to this one:
apps/training/solutions/dev/audio_player/audio_player_lab5.
In MHGC, open the Image Assets screen by clicking the Asset tab and selecting Images.
Select each image individually by clicking on the file name from the Icons folder. Add all 3 images to assets.
Keep the default settings for each image except for MCHP_LOGO.bmp. For this image, we want to mask the background color that is not part of the logo design. Check the Color Masking box and set the RGB values of the Mask Color to 29,46,60. This happens to be the RGB values of the background in the image.
Step 3.3: Import Fonts
In this step, we will add two system fonts: Arial14pt and Arial18pt.
In MHGC, open the Font Assets screen by clicking the Asset tab and selecting Fonts.
Select the icon to Add Installed Font. This will bring up a screen that allows you to select a font installed in your system. Select the Arial font and press Ok.
This will bring up a screen that allows you to configure different properties of the font. Select the Style tab and choose a size of 14.
Now, select the Glyphs tab and click the add icon to add a glyph range. For Arial14pt, we want the Basic Latin range of characters, number 32 to 127. Then press the Ok button.
Finally, we want to rename this font from the default name to Arial14pt. Click on the font and select the Rename Selected Font icon. Then input Arial14pt and press Ok.
Similarly, repeat the above steps to add the Arial18pt font. Except for the range values, we only want to add one character, a caret. This will be used by the ScrollUpButton.
Select size of 18.
Select a starting and ending character of 94. This is the caret character (^).
Rename the font to Arial18pt.
Step 3.4: Create Color Schemes
Create color schemes to standardize the look and feel of the user interface. These will be used for the display widgets created in the "Add Widget Objects" step. You can add color schemes in the Schemes window by clicking on the Add icon.
Add the color schemes listed below. Change the name field by typing over the existing name field. The color can be changed by clicking the … icon to the right. This will bring up a color selection screen that will allow you to enter the RGB values.
Step 3.5: Add Text Strings
In MHGC, open the String Assets screen by clicking the Asset tab and selecting Strings.
Select the icon to Add New String. This will bring up the Add String Dialog box. Enter the name of the string. This is a name used to reference the string and not the string value itself.
Now, we can enter the actual string text. Click in the value field to enter the text. Then click in the font field to get a dropdown menu of available fonts.
Do this for each of the strings listed in the table below. Make sure that the ScrollUp string has the
Arial18pt font.
Name |
Value |
Font |
GroupBoxTitle |
Mode |
Arial14pt |
NoFilesMessage |
No Audio Files Found. Please Select Reader Mode to Load. |
Arial14pt |
RadioButtonPlayer |
Player |
Arial14pt |
RadioButtonReader |
Reader |
Arial14pt |
ScrollDown |
v |
Arial14pt |
ScrollUp |
^ |
Arial18pt |
TitleString |
Harmony SD Card Audio Player / Reader |
Arial14pt |
Once complete, you shoud have all of the strings listed in the image below.
Step 3.6: Create Static Logo and Title Fields
In MHGC, click on Layer0 in the Tree View window.
This should bring up the properties for Layer0 in the Properties Editor window. Change the color scheme to BackgroundScheme and check the Transparency Enabled box.
Add a gradient box to highlight the logo image. Click and drag the Gradient widget from the Widget Tool Box window onto the screen image. Then, use the Properties Editor window to specify the detail placement and properties of the widget.
Add an image box to display the logo image. Click and drag the Image widget onto the screen. Change the properties as specified below. In the Image field, select the graphics image imported from the step above.
Add a text box to display the application title. Click and drag a Text Field widget onto the display screen. Change the properties as specified below.
Add a text box to provide the ability to display an error message. Click and drag a Text Field widget onto the display screen. Change the properties as specified below.
Step 3.7: Create Radio Button Group Box
Add a group box for the radio buttons. Click and drag a Group Box widget onto the display screen. Change the properties as specified below.
Add a Radio button for Player mode. Click and drag a Radio Button widget onto the display screen. Change the properties as specified below.
For this widget, we need to add action code that will be executed when the button is selected. Click on the … icon next to the Selected field. This will bring up a screen where you can add the action code text shown below.
Add another Radio button for Reader mode. Click and drag a Radio Button widget onto the display screen. Change the properties as specified below.
Similar to the previous radio button, add the action code shown below to the Selected field.
Step 3.8: Create Play List Box
Add a list box to display the list of files from the SD card. Click and drag a List widget onto the display screen. Change the properties as specified below.
Click on the … icon next to the Selection Changed field. This will allow you to add action code when the list box selection changes.
Add a scroll up button for the list box. Click and drag a Button widget onto the display screen. Change the properties as specified below.
Add action code for the button Released action.
Add a scroll down button for the list box. Click and drag a Button widget onto the display screen. Change the properties as specified below.
Add action code for the button Released action.
Step 3.9: Create Volume Control Items
Add a rectangle border to go around the volume slider. Click and drag a Rectangle widget onto the display screen. Change the properties as specified below.
Add the volume slider to change the output volume from minimum to maximum. Click and drag a Slider widget onto the display screen. Change the properties as specified below.
Add action code to control input from the volume slider as shown below.
Add volume button that can mute and unmute the sound. Click and drag a Button widget onto the display screen. Change the properties as specified below.
Add action code to unmute the sound (button pressed) as shown below.
Add action code to mute the sound (button released) as shown below.
The final screen image should appear as shown below.
Step 4: Generate Harmony Code
Expand the source file folders in the Projects pane. Notice that these are the files from Lab 4 that you copied to this directory.
Click on the Generate Code button as shown in the following figure. Notice the Merging Strategy option.
Note the new decoder folders added to the lab.
Check for Harmony 3 issues in previous released versions.
In previous released versions of the Harmony 3 graphics code (gfx 3.4.0 and before), there were two issues that needed to be resolved in drv_maxtouch.c and drv_gfx_lcc.c. The directory structure below shows the location of these two files which gets generated by MHC.
In the source file drv_gfx_lcc.c, make sure the parameter in the call to DMAC_ChannelCallbackRegister (line 306) is DRV_GFX_LCC_DMA_CHANNEL_INDEX.
Also, In the source file drv_maxtouch.c, make sure the parameter in the call to pDrvInstance->drvOpen (line 875) is DRV_MAXTOUCH_I2C_MODULE_INDEX.
If you do need to modify these lines, the next time you generate code, you will see the following merge screens displayed. Just press the Close button.
If you were to build the project now, the build would fail. The custom display routines added in the Graphics Composer have not yet been defined in the source code. You will add the application display code to implement this API in the next step.
Step 5: Include Application-specific Source Files, Add Required Code, and Build the Project
The app_button_press_task.c and app_button_press_task.h files are no longer needed as their functionality will be replaced by the radio buttons on the display. Right click on these files and select Remove From Project.
Copy source files into your project's Source Files folder:
- app_display_task.c
- app_display_task.h
Copy them from this folder:
apps/training/solutions/audio_player/audio_player_lab5/dev_files
to this one:
apps/training/solutions/dev/audio_player/audio_player_lab5/firmware/src.
Add the copied source files to your project.
- Add app_display_task.c to the Source Files/app folder (in the MPLAB X IDE Projects pane) by right clicking and selecting Add Existing Item…
- Add app_display_task.h to the Header Files/app folder by right clicking and selecting Add Existing Item…
The files under the project should look like this:
As you did in the previous labs, modify the app.c and app.h files from audio_player_lab4 to replace the Button task related API with the Display task related API in APP_Initialize and APP_Tasks functions.
Also, modify app.h to include app_sdcard_audio_task.h and app_display_task.h.
Now that you have added graphics to this lab, the SD card audio task needs to notify the display task whenever the tracks list is updated. Open the app_sdcard_audio_task.c file and add the following function call under the APP_SDCARD_AUDIO_CARD_STATE_INIT state.
Save all the files before closing.
Now, you are ready to build the code. Click the
Clean and Build icon and verify that the project builds successfully.
Step 6: Review the Application Code
Application File: app_sdcard_audio_task.h
This file remains the same as in audio_player_lab4. No changes are needed. Refer Review the Application code section in Lab 4.
Application File: app_display _task.h
Open file app_ display _task.h (in the MPLAB X IDE Projects pane, under Header Files > app folder). This file defines application enums, data, and APIs.
The volume mute enumerator helps manage the current audio activity and acts (on a button press) to mute and un-mute, or control volume increase and decrease.
The app mode enumerator helps manage the mode selected by the user on the UI.
The Display data structure holds the volume/mute information, the ID of the current track being played, and a handle to system timer for implementing de-bouncing for the volume slider.
Application File: app_sdcard_audio_task.c
This file remains the same as in audio_player_lab4, except for a function call that is added to notify the display task of any changes to the track list. No other changes are needed. Review the Application Code section in Lab 4.
Application File: app_display_task.c
Open file app_display_task.c (in the MPLAB X IDE Projects pane, under Source Files > app folder). This file contains the display task initialization and task functions.
The function APP_DISPAY_Initialize sets up the default value for the volume/mute variable as not changed.
The function APP_DISPLAY_Tasks performs the following tasks:
Once the display screen has been drawn by the gfx library state machine, it initializes the list box object with tracks from the SD card, whenever the track list is updated by the SD card audio task. It also sets the default volume level in the volume slider.
It handles changes to the application mode. When the user switches from the SD card player mode to USB SD card reader mode, it suspends the streaming, attaches the USB device and then switches the application mode to reader mode. When the user switches back to SD card player mode, it reinitializes the SD card audio task, detaches the USB device, and then switches the application mode to player mode.
It handles the volume and mute control. The user can increase or decrease the volume through the volume slider. For mute control, the volume slider is disabled or enabled based on whether the volume is mute or un-mute. Also, the audio CODEC is updated with the new volume level. Note that the volume slider has de-bounce logic implemented by registering a call-back with the system timer service. This avoids having multiple I²C transactions to the audio CODEC for volume changes.
It updates the list box object with the current audio track being played. The audio files are played one after the other from the SD card, and hence, when the current audio track changes, the list box object’s focus is updated to reflect the current (changed) track being played.
The function _APP_DISPLAY_HandleMuteOn handles the volume mute event by setting the level of the volume slider object to zero and then disabling the volume slider object.
The function _APP_DISPLAY_HandleMuteOff reverses the action by setting the level of the volume slider object to its previous position and then enabling the volume slider object.
The function _APP_DISPLAY_HandleVolumeIncreaseDecrease scales the level of volume slider object to the volume level of audio CODEC.
Step 7: Debug Your Application
Congratulations! You are ready to DEBUG
your fifth application!
Before you program and run our application, make sure that the micro SD card has at least one audio file saved on it.
Make sure that the micro SD card is inserted into the micro SD card slot (J8) on the MEB II.
Before you start the debugger, you may want to set a breakpoint in the application file in app_display_task.c to verify that important stages of the application are executed successfully.
Put a breakpoint in APP_DISPLAY_Tasks which will indicate that the graphics library has finished drawing the current screen and that the application is now ready to populate the list box object with the track list.
To gain a better understanding of graphics objects event handling and how the custom actions are executed, you may want to put breakpoints in the processEvent function (liberia_event.c file).
Debug your application! Click the
Debug Main Project icon.
Remove all breakpoints and press key F5 to allow the application to run.
You should see the display now. Notice that the display looks exactly the same as MHGC's simulated display.
Experience the different features of the user interface:
- Change the application mode from 'player' to 'reader' by selecting the radio buttons.
- Press the scroll up/scroll down buttons to navigate through the track list.
- Increase/decrease the volume by changing the volume slider position.
- Mute/un-mute the volume by pressing on the volume mute button.
Connect a headphone to the HP Out connector on the MEB II and listen to your favorite music. Notice the track ist will scroll as the tracks get played out one after the other.
Results
If everything worked well, you should see a graphical user interface on the MEB II display, populated with a list of tracks read from the SD card along with volume and mute control. The user interface should respond to touch events and you should be able to select random tracks for playing, increase or decrease volume level, and mute the audio output. You should be able to put the application either into audio player mode or SD card reader mode by selecting the appropriate radio button on the UI. You might not get the intended results if the Graphics Library was not configured correctly, an invalid Graphics Display was selected or if the Touch interface was not configured correctly.
Analysis
In this lab, you added graphics display support. You used the MHGC, embedded within MHC, to design and develop the graphical user interface. You added a display driver and touch driver from the list of driver implementations provided by MHC. You added custom actions to handle various UI events like touch, pressed, released, etc. At the application layer, you added a new task (state machine) to handle graphics related activities like populating the display with the list of tracks read from the SD card, handling track change events, volume increase/decrease, and mute events.
Conclusions
With this lab, you add graphics support to your application and extend it a step further. The lab demonstrates the use of the MHGC to design and develop UI screens. It shows you how to add and customize different properties of widgets, create schemes, add images and font libraries, and how to handle events originating from the widgets. The lab shows how easily you can add complex graphics, display drivers, and touch screen drivers to your project using MHC. Going forward, you can experiment with MHGC and try to create a UI of your choice. You may want to add multiple screens!