Draft proposal of standard parameters and functions for DisplayIO Widgets

[kmatch98]

The DisplayIO_Layout library will rely on having several standard parameters for any "widgets" so this function can place them in the desired layout. Additionally, for any touch-related objects, it is useful to have standard response functions.

The starting point for this proposal was the Adafruit_CircuitPython_Display_Text library with the label item along with the Adafruit_CircuitPython_Display_Button library with the button item.

I'd especially like proposals on the standardized "Response functions" for widgets.

---

Proposed widget parameters and naming:

• Position and sizing
  • x, y : upper left corner of the widget, in display pixel coordinates (see diagram)
  • anchor_point : (a, b) two values between 0 and 1. Widget placement using anchor_point and anchored_position should operate the same as display_text.Label (see candy hearts example: https://learn.adafruit.com/circuit-python-tft-gizmo-candy-hearts/how-it-works)
  • anchored_position : (C,D) in pixels (see anchor_point description above)
  • width : in pixels
  • height : in pixels
  • bounding_box : (x, y, width, height), *getter only
• Touch-related
  • touch_padding : in pixels, additional space around the bounding_box that accept touch input (alternately, modify touch_boundary directly)
  • touch_boundary : (x, y, width, height) region that will accept touch input
• Response functions and data
  • contains(touch_point) : responds True if touch-point is within the widget’s touch_boundary
  • selected(touch_point) : widget was just touched, reaction to selection will depend upon the widget’s needs
  • still_touched(touch_point) : widget remains touched (***?)
  • released(touch_point) : widget is released (***?)
  • value : type will depend upon the widget (getter/setter required)
• Widget naming (*** Should each widget have a label that can be optionally displayed and positioned?)
  • label : text label to describe the button
  • display_label : Boolean, set True to display the name of the widget
  • label_font : font
  • label_anchor_point_on_widget : proposed (*need to add description and diagram)
  • label_anchor_point_on_label : proposed (*need to add description and diagram)
• Optional parameters and naming recommendations:
  • _color : use as many parameters as needed for fill and outline colors for the widget structures, prefer to include _color somewhere in the parameter name. Should accommodate hex 0xFFFFFF 8-bit RGB color codes, but prefer to also support tuples (R, G, B).
  • _stroke : this is the width in pixels of outline features. Use as many as needed, prefer to include _stroke in the parameter name.
  • animation_time : in seconds, the time required for the widget’s animation. For example, for a switch, the time allotted for switching from off to on and vise versa. Preferably, the widget will check the elapsed animation time and redraw accordingly, so that the response time is the approximately the same even when run on different processors and systems. (see example in switch_round_horizontal.py, see selected function)
  • orientation : horizontal or vertical
  • flip : Boolean, default is False. Set True if mirror image is desired.

Coordinate scheme

Here is a graphic with a proposal of some of the naming of the pixel coordinate schemes:

Draft code snippet: Switch widget

Here is a code snippet of the __init__ function for a draft of a switch widget that runs on the Adafruit PyPortal:

class SwitchRoundHorizontal(displayio.Group):

    def __init__(
        self,
        *,
        x=0, # Placement of upper left corner pixel position on the display
        y=0,
        width=None, # defaults to the 4*radius
        height=30,
        name="",
        value=False,
        touch_padding=0,
        anchor_point=None,
        anchored_position=None,
        fill_color_off=(66, 44, 66),
        fill_color_on=(0,100,0),
        outline_color_off=(30,30,30),
        outline_color_on=(0,60,0),
        background_color_off=(255,255,255),
        background_color_on=(0,60,0),
        background_outline_color_off=None, # default to background_color_off
        background_outline_color_on=None, # default to background_color_on
        switch_stroke=2,
        text_stroke=None, # default to switch_stroke
        display_button_text=True,
        animation_time=0.2, # animation duration (in seconds)
        ):

        super().__init__(x=x, y=y, max_size=3)

Widget positioning

As described above, the widget position on the screen can be defined directly by widget.x and widget.y or by the combination of widget.anchor_point and widget.anchored_position.
Here is a reference image for the usage of anchor_point and anchored_position as snipped from the Adafruit Learn guide's Candy Hearts Example:

[makermelissa]

What if all of these widgets used a "Control" or "Widget" superclass rather than directly deriving from Group and the superclass handled these standard events?

[FoamyGuy]

@kmatch98 this is a really great start! Thanks for taking the time to get all of this down.

I am in agreement with the majority of what you've laid out. I'll list only the things I might add to or suggest different below.

I am thinking the same as Melissa as well about putting these things into a class and then having the specific widgets extend this instead of Group. I'll refer to it as "Control" here, I do like that name, but open to other ideas still as well.

For the response functions I wonder if there may be a way to leverage the contains() function along with the debouncer library to get the other property states concepts like just touched and just released.

Widget naming - I don't think a widget_label option should be done at the Control level. I do see the usefulness of this construct, but I think the way to go about it is making a specific widget / layout. Like LabeledThing class (hopefully better name) and it has a set_thing(control_obj) function. So it works a bit like the GridLayout (or any layout) in that you "add" or in this case "set" a control into it and it adds "something" to the control. This one will add the text off the side (or wherever configured) of the control. This way it will be still be able to work with all controls, but doesn't need anything else specific to exist on the Control class for it to work.

animation_time I'm not sure if we need this on Control. Some widgets may not need it, and I'm not sure if the layouts would need to access it.

orientation Same for this one. It could live on the specific widgets need it. As long as they handle width and height appropriately based on their orientation the layouts can use that and not have to worry about orientation.

flip is interesting. Is the use case for rear-projection, or mirrored screen? I'm not sure if we have an easy way to achieve it in disiplayio, but perhaps I'm unaware or something.

Coordinate scheme - This is an awesome illustration! Would love to see diagrams like this find a home in the documentation or learn guide.

[kmatch98]

Thanks for the encouragement.

I’m not sure I follow the difference in purpose or function between a Group and a Control class, so I could use some clarification on how this would be organized.

Right now, the only thing that can be displayed using display.show() is a Group class element. And currently there are only a few things that can be held in a Group, I think only Group and TileGrid. What level would this “control” class fall into?

If y’all can sketch a block diagram of your proposal that will help me understand better.

@FoamyGuy thanks for the comments on the variables, I think some variables should be categorized as “mandatory” (so things like grid_layout can operate on them) and other variables as “optional”. And anytime we can use meaningful reuse of variable names that will probably make life easier, even for optional variables.

Also I like your idea about the labeling of a widget. I struggled to make sense of a good way of doing that, but I think you are on a good track about having a separate “thing” that does any labeling. I’m also thinking of the best way of putting text inside of a widget, like numbers on a dial and how it can use a common way of laying them out. Maybe that is the same as a “grid layout” task, but used inside a widget.

Regarding widget orientation, taking the example of a switch I could imagine someone wanting it to switch right/left (horizontal) or up/down (vertical with ON is up), then next someone will want eventually want the mirror image of that (vertical but ON being down), so I thought maybe “flip” would be a way to describe the mirror image orientation. (I think I saw that lingo used some other library, maybe for setting up matrix display?). Another example, if someone wants a Sparkline to scroll in the opposite direction, they could “flip” it. I was thinking the individual widget drawing routine could be designed to handle dealing with the orientations (if that feature is desired).

Thanks again for the feedback. I’ll do some more cleanup on the slider switch widget example (better color handling) and post it somewhere for general consumption.

[FoamyGuy]

I am imaging it like this:

The main reason for Control existing is that Group itself doesn't have width and height or things that rely on them like anchor_point and anchored_position. But it also gives us a nice place to define the common interface that all of the layouts and other "enchancing" widgets like the LabeledThing will assume exists in order to do whatever they need.

[FoamyGuy]

Ah I think I see what you mean with flip, its more conceptual and the specifics can be up to each widget, not necessarily strictly mirroring the pixels. I do think that is a nice option to have.

Orientation I think I understand what you mean better as well. But I do think it can live on each widget that wants to have it. Doesn't need to be on Control

[FoamyGuy]

In the initial code I have a widgets directory that I'm thinking we can include things like the RoundSwitch that you made if you are interested in having it there.

[makermelissa]

That's the great thing about using a superclass. The subclasses can override any functions from the superclass for any specific implementation details.

[TG-Techie]

I agree with @makermelissa and @FoamyGuy on separating a "Control" or "Interactable" base widget. It also makes that functionality opt-in as opposed to required for every "Group", might be a ram/space saver on some of the more ram tight m4 boards (like the Nrf). Also groups don’t have dimensions where as Controls do is a great point.

There may be a be advantages to separating controls further? Not all widgets may need control. It could be something like Group -> Widget -> Control. Widget adds dimension and Control adds, well, control.

If I may add in two cents to the original post. Might I suggest that 'x' and 'y' be specified together as 'coord=', a tuple of (x, y). Same thing for width and height as (width, height) as 'dims='.
From my experience having the user specify them as pairs allows the backend more flexibility to reason about where it should be, and generally it means the user can't input invalid or implicit data.

For the data/control interface I'd suggest separating selection from pressing from updating scroll coordinates, etc.
As a small case example, I've had to decouple selection, pressing, and "scrolling" from each other as pressing and scrolling are mutually exclusive but a scroll may start with a finger on a button/selectable. when scrolling starts the button needs to be deselected without activating it's functionality. (That reasoning is handled by an input event loop)

Another useful debug widget may be frame, something to draw the exact borders around a widget.

[kmatch98]

I'm trying to digest y'all's inputs so I made a chart to see if I'm getting it right. Also, I'm unsure how generic the Control class should be so I thought I'd make a graphic to see if I understanding it:

Here are some questions:
Should the Control class be it's own separate Class, or a subclass of Widget?
How generic do we want to make Control? I first envisioned it as related to dealing with touch screen inputs, but it could be more generic. For example, do we want the Control class to be able handle D-pad or joystick switch inputs to control a cursor widget on the screen?

Specifically regarding touch_screen related control functions, it seems like touch_screen capabilities differ a lot depending upon the type. The current Adafruit_CircuitPython_Touchscreen Library for the resistive touch screen seems to only provide single touch response. But I guess there could be added wrappers to provide touch_down, still_touched and touch_up actions. Also, some touch screens also provide other gesture response (multi-touch points, pinch zoom, etc.). How should this Control class be configured? How many Control functions should be required functions, or does the event loop just dump a Dictionary of touches and/or gestures to the Control, and each Control has to decide if it responds to those?

One more question about contains(). I was first envisioning that the event loop will check each widget using contains() and then call the widget.selected() function. In contrast, should the event loop just send the Touch/Gesture event dictionary to each widget and then they each return whether they handled that event?

As I'm fairly new to all of this, so I'm mixing up a lot of questions and concepts between the class structure and the event loop handling, so I appreciate your patience. Also, I'm sure this has been done before, so if you have a suggested reference design, please fire it over.

[TG-Techie]

As I have this on hand, Here is an event loop I've implemented. It scans through all the widgets that have certain control functionalities and then informs them of what to do. That way the widgets don’t have to parse raw data*

in this design widgets' control functionality is defined by the presence of specific methods but I could easily be defined as subclasses with overrides.

[*] scrolling is not finished, so the update coord does have to for now
(It is lacking exclusive either press or scroll behavior atm)

https://github.com/TG-Techie/TG-Gui-Std-CircuitPython/blob/main/tg_gui_std/event_loops.py

[kmatch98]

I updated the sliding switch widget now with classes as laid out in the diagram above. Here are the main updates:

Demo files are here, along with a pyPortal example.

To Clarify:

• An input parameter to WidgetLabel is a Widget. It uses the text from Widget.name and Widget.bounding_box to define the placement and adds the WidgetLabel directly to the Widget group for display.

This is my first time making a more complicated set of interacting Python classes, so I welcome any suggestions that you have to improve this. It took me quite a bit of trial and error to figure out the use of *args and **kwargs along with the super().init details. If I can improve how to do these, I'm eager to learn how.

I appreciate your inputs!

---

The diagram above captures the general Class design. But here it is in a list, along with a few comments.

SwitchRoundHorizontal class:

• Subclass from Widget and Control classes (uses multiple inheritance)
• Includes a WidgetLabel class as an option to label a switch.

Widget class:

• Positioning
• Sizing
• Name

Control class: Outlines response functions

• contains(touch_point): for touch checking
• selected
• still_touched (needs a better name)
• released
• gesture_response: for future expansion if gestures are available

WidgetLabel class: Positions the label with relative position on the widget.

• font
• Widget - the widget that the label should be added to, uses the text from Widget.name and Widget.bounding_box to calculate the WidgetLabel position
• anchor_point
• anchor_point_on_widget

This class positions the label's anchor_point at the widget's anchor_point_on_widget.
Note that X and Y values for anchor_point_on_widget can be < 0.0 or can be > 1.0 to place it outside of the widget's bounding_box.

[kmatch98]

I submitted a draft pull request with the initial version of the proposed class definitions and an example.

Feedback is welcome.

[kmatch98]

This is superseded by the PRs and other discussions so I’m closing this.