Tag: IFML

A Model-Driven Approach for Multi-experience Development Platforms

by Marco Brambilla, posted in IoT, modeling, software engineering

Modern User Interfaces (UIs) are becoming complex software artifacts themselves, through integration of AI-enhanced software components that enable even more natural interactions, including the possibility to use Natural Language Processing (NLP) via chatbots or voicebots (aka., Conversational User Interfaces or CUIs).

Some times, several types of UIs are combined as part of the same application (e.g. a chatbot in a web page), what it is known as Multiexperience User Interface. These multiexperience UIs may be built together by using a Multiexperience Development Platform (MXDP).

“Multiexperience development involves ensuring a consistent user experience across web, mobile, wearable, conversational and immersive touchpoints”. [Gartner]

A typical scenario of multiexperience user interaction could unroll as follows (see image below too). Suppose that a customer on a Sunday morning wants to buy a new technical product (a cell phone or a home theater system). He first interacts with his home assistant (like Alexa or Google assistant) to ask it to find the best nearby tech store open on Sunday. With this information in mind, he looks at the store web site on his PC and, being satisfied with the kind of store, he asks the web site chatbot to find the type of products he is looking for. After browsing the various alternatives, he finds one item he likes, and sets the place and the product as preferences on his mobile phone. He reads the details of the product on the phone while walking to his car. When he reaches the car, he transfers the information about the place to the car navigation system and drives there. Finally, in the stores he looks around, tries various items, reads the reviews about them on a dedicated mobile app, and finally picks up the product and pays for it.

This kind of dynamic and seamless interaction demands a variety of complex design and implementation mechanisms to be put in place. Clearly, also very critical integration, evolution, and maintenance challenges need to be faced for these CUIs. Developers need to handle the coordination of the cognitive services to build multiexperience UIs, integrate them with external services, and worry about extensibility, scalability, and maintenance.

We believe a model-driven approach for MXDP could be an important first step towards facilitating the specification of rich UIs able to coordinate and collaborate to provide the best experience for end-users. Indeed, most non-trivial systems adhere to some kind of model-based philosophy, where software design models (including GUI models) are transformed into the production code the system executes at run-time. This transformation can be (semi)automated in some cases.

Our recent research tackles the application of model-driven techniques to the development of software applications embedding a multiexperience UI.

The research has been published in our recent paper Towards a Model-Driven Approach for Multiexperience AI-based User Interfaces, co-authored by Elena Planas, Gwendal Daniel, Marco Brambilla and Jordi Cabot, recently published in the International Journal on Software and Systems Modeling (SoSyM) available online here (open access).

The paper contribution is twofold:

  • we raise the abstraction level used in the definition of this new kind of conversational and smart interfaces.
  • we show how these CUI models can be used in conjunction with more “traditional” GUI models to combine the benefits of all these different types of interfaces in a multiexperience development project.

In practice, we propose a new Domain Specific Language (DSL), that generalizes the one defined by the Xatkit model to cover all types of CUIs, and we show how this seamlessly integrates with appropriate extensions of the IFML model to design comprehensive multi-experience interfaces.

IFML model integrating traditional navigation of a web interface and a chatbot component.

You can refer to the full paper here for covering the details. The paper reference is:

Planas, E., Daniel, G., Brambilla, M., Cabot, J. Towards a model-driven approach for multiexperience AI-based user interfaces. Software and System Modeling (SoSyM) 20, 997–1009 (2021). https://doi.org/10.1007/s10270-021-00904-y

(open access, CC-BY license)

Model-driven Development of User Interfaces for IoT via Domain-specific Components & Patterns

by Marco Brambilla, posted in event, IoT, modeling, software engineering

This is the summary of a joint contribution with Eric Umuhoza to ICEIS 2017 on Model-driven Development of User Interfaces for IoT via Domain-specific Components & Patterns.
Internet of Things technologies and applications are evolving and continuously gaining traction in all fields and environments, including homes, cities, services, industry and commercial enterprises. However, still many problems need to be addressed.
For instance, the IoT vision is mainly focused on the technological and infrastructure aspect, and on the management and analysis of the huge amount of generated data, while so far the development of front-end and user interfaces for IoT has not played a relevant role in research.
On the contrary, we believe that user interfaces in the IoT ecosystem they can play a key role in the acceptance of solutions by final adopters.
In this paper we present a model-driven approach to the design of IoT interfaces, by defining a specific visual design language and design patterns for IoT applications, and we show them at work. The language we propose is defined as an extension of the OMG standard language called IFML.

The slides of this talk are available online on Slideshare as usual:

Automatic Code Generation for Cross-platform, Multi-Device Mobile Apps. An Industrial Experience

by Marco Brambilla, posted in data science

With Aldo Bongio (WebRatio), Jordi Cabot (ICREA and UOC), Hamza Ed-douibi (EMN) and Eric Umuhoza (Politenico di Milano), we worked on a research on Automatic Code Generation for Cross-platform, Multi-Device Mobile Apps.

We presented our study at the MobileDeLi workshop, where we reported on a comparative study conducted to identify the best trade-offs between different automatic code generation strategies.
Here are the slides presented there:

We covered the following strategies by implementing them using different technologies and target platforms:

  1. PIM-to-Native Code (NC)
  2. PIM-to-PSM-to-NC
  3. PSM-to-NC.
  4. PIM-to-Cross Platform Code (CPC)
  5. PIM-to-Framework Specific Model (FSM)-to-CPC

Some additional details are available in this post by Eric on Jordi’s blog.

Our study showed that there is no approach better than others in absolute terms but provided useful guidelines (e.g. cross platform approaches are generally advisable for companies with limited resoures) that helped us to identify the best strategy for the WebRatio company in particular.

Obviously, further investigations are ongoing…

To keep updated on my activities you can subscribe to the RSS feed of my blog or follow my twitter account (@MarcoBrambi).

OMG Standards At Work in the Industrial Internet of Things (IoT) – summary of the day and materials

by Marco Brambilla, posted in data science

Today September 24, 2015, a special event took place in Cambridge, MA: the OMG Standards At Work in the Industrial Internet of Things (IoT).

I have been asked to give a speech on the role of user interaction modelling within complex IoT system in industry: Internet of Things and Internet of People: The Role of User Interaction in the IIoT vision.

The full slide deck is available here:

 

Here is my take on the topic: User interaction plays a crucial role in every system. This is true for IoT too. Sensors, actuators and intelligent things connected together can cooperate and exchange information, but their ultimate goal is to provide value to people. Such value can be perceived only through appropriate user interfaces, which visualise information (through dashboard, reports, or infographics), let user navigate the information, and also interact with the devices, by setting properties or regulating their behavior. That’s why it’s important that in the IIoT development context we consider also user interaction. In my presentation I introduced IFML, the Interaction Flow Modeling Language, the OMG standard that focuses on user interfaces and their integration with information systems, data sources, sensors and actuators. The presentation reports on some success stories from the industry, where IFML has been successfully applied. Large scale examples include consumer-oriented user interfaces, backend systems, data analysis dashboards, and interactions for command and control. Adopters include GE, Acer Computer, banks, utilities and military.

The other speakers of the event also provided interesting use cases, examples and insights on the span of the effort and potential of IoT, especially in the industrial environment. The speeches were as follows:

  • Welcome: by Dr. Richard Soley, Chairman and Chief Executive Officer, Object Management Group (OMG) and Executive Director, Industrial Internet Consortium (IIC) —
  •  Introduction and Overview: by Andrew Watson, Vice President and Technical Director Object Management Group (OMG)
  • DDS Applications in the Industrial Internet of Things: by Dr. Stan Schneider, CEO, Real-Time Innovations, Inc.
  • DDS – Aligning OT and IT to Deliver the Potential of the Industrial Internet-of-Things (IIoT): by Steve Jennis, Sr. Vice President, Corporate Development, PrismTech
  • SysML – System Modeling Language Benefits for the Complex Systems of IIoT: by Matthew Hause, GTM Technical Specialist, Engineering Fellow, PTC, OMG UPDM Co-Chair
  • CISQ – Software Risk in the IoT Universe: by Dr. Bill Curtis, Director, Consortium for IT Software Quality (CISQ)
  • System Assurance and Related Standards: by Dr. Ben Calloni, Lockheed Martin Fellow, Lockheed Martin and Co-Chair OMG System Assurance Task Force

Find the full program of the day and materials by the other presenters here: OMG Standards for IIoT agenda.

To keep updated on my activities you can subscribe to the RSS feed of my blog or follow my twitter account (@MarcoBrambi).

OMG Industrial Internet of Things (IIoT) Standards At Work

by Marco Brambilla, posted in data science

What do SysML™, IFML™, SysA and DDS™ standards have in common, and how can they be put at work in the (Industrial) Internet of Things [IoT and IIoT] context?

Industrial Plant schema
Industrial plant with IoT sensors and data sharing
To respond to this question, I will join a special event by the Object Management Group (OMG) on September 23, 2015, from 1:30 pm – 5:30 pm at the Cambridge, MA meeting.
The event will consist of short presentations of the standards, panels and discussions of industrial cases and will allow a closer look at how OMG standards are shaping the Industrial Internet of Things around the globe. OMG is joining forces with IIC (Industrial Internet Consortium) for covering IIoT.
The Industrial Internet of Things (IIoT) is delivering improved productivity, major cost savings, and streamlined processes to professionals from all industries. OMG has been active in IIoT standardization from long before “IIoT” became an industry buzzword.
This half-day information session brings together industry experts who will share case studies of these standards at work in the Industrial Internet, and present their vision of the future within this rapidly growing field. OMG is committed in supporting IIoT efforts and its standards cover several aspects of the field, as described here.
The topics covered during the event include:

  • SysML – System Modeling Language. SysML is a dialect of the Unified Modeling Language (UML) standard, and supports the specification, analysis, design, verification and validation of a broad range of systems and systems-of-systems.
  • IFML – Interaction Flow Modeling Language. IFML is designed for expressing the content, user interaction and control behavior of the front-end of applications, including complex systems found in the Industrial Internet of Things.
  • SysA – System Assurance and CISQ. OMG’s Systems Assurance Task Force (SysA TF) works with CISQ on standards that ensure the reliability, safety and security of IIoT systems.
  • DDS – Data Distribution Service for Real-Time Systems. OMG’s  DDS standard provides a protocol that meets the demanding scalability, performance, and Quality of Service requirements of IIoT applications spanning connected machines, enterprise systems, and mobile devices.

The event is free and anyone can join. You can register to the event here:

OMG IIoT event
You can find here the official page of the event, with the detailed program and other information.

To keep updated on my activities you can subscribe to the RSS feed of my blog or follow my twitter account (@MarcoBrambi).

Video of the Interview with Richard Soley (OMG) on IFML and user interaction modelling

by Marco Brambilla, posted in data science

Here is a short clip of the interview regarding the Interaction Flow Modeling Language (IFML) recorded in March 2015, in occasion of the release of IFML 1.0.
In the interview we discuss with Richard Soley about the relevance of user interaction modelling, the way it can be integrated with broader modelling projects, and the impact it has on overall design effort of software systems. Emanuele Molteni also discusses some success stories in the application of IFML in large-scale industrial projects in the US, by means of the WebRatio tool.

https://www.youtube-nocookie.com/embed/ZT1Z0zOrOc4
To keep updated on my activities you can subscribe to the RSS feed of my blog or follow my twitter account (@MarcoBrambi).

Here we are: IFML 1.0 published by the OMG officially

by Marco Brambilla, posted in data science

IFML 1.0 is finally published!

The OMG released the official 1.0 version of the new standard in March 2015. It took several years of research, discussions and validation, but here we are with the standard specification. The specification document can be downloaded for free at:

http://www.omg.org/spec/IFML/

Here is Emanuele ready for the presentation given at the Object Management Group ADTF (Analysis & Design Task Force).

Emanuele Molteni presenting IFML implementation
and success stories at the OMG meeting

With perfect timing, we are also ready to go to the market with:

To keep updated on my activities you can subscribe to the RSS feed of my blog or follow my twitter account (@MarcoBrambi).

Keynote speech on User Interaction Modeling at Modelsward 2015 in Angers

by Marco Brambilla, posted in modeling

On February 10, 2015 I gave a keynote at Modelsward in Angers, France.

The speech focuses on the modeling of software UIs through graphical domain-specific languages and in particular shows the new standard adopted by OMG called IFML (Interaction Flow Modeling Language) at work. My presentation illustrates the basic concepts of IFML, presents the design best practices and integration with other modelling languages, and discusses some large-scale industrial experiences (also featuring quantitative measures of productivity) achieved through IFML and associated full code generation techniques.

The full video of my presentation (1 hour long, if you can endure it!) is available on Vimeo thanks to the Insticc service. See it here too:

The IFML book – OMG’s Interaction Flow Modeling Language explained

by Marco Brambilla, posted in data science

After almost one year of work, the result of our efforts finally came to light in late December 2014. Since it was almost Christmas time, we decided to wait 2014 for the launch. But now, here we are.
OMG Press and Morgan Kauffman published our book:

Interaction Flow Modeling Language:
Model-Driven UI Engineering of Web and Mobile Apps with IFML
Additionally, I can announce officially we will have a launch event at the next OMG meeting in Reston, VA, USA, in March 2015.
The book introduces the reader to the novel OMG standard Interaction Flow Modeling Language (IFML). Authors Marco Brambilla and Piero Fraternali are also authors of the IFML standard and wrote this book to explain the main concepts of the language. They effectively illustrate how IFML can be applied in practice to the specification and implementation of complex web and mobile applications, featuring rich interfaces, both browser based and native, client side components and widgets, and connections to data sources, business logic and services.
The book provides you with unique insight into the benefits of engineering web and mobile applications with an agile model driven approach. Concepts are explained through intuitive examples, drawn from real-world applications. The authors accompany you in the voyage from visual specifications of requirements to design and code production. The book distills more than twenty years of practice and provides a mix of methodological principles and concrete and immediately applicable techniques. Dr. Richard M. Soley, chairman of the OMG, wrote the foreword of the book.

You can buy the book in paperback (on Amazon or any other bookstore) or electronic format (Kindle on Amazon; PDF e-book on Elsevier store).

If you are looking for some basic introduction to model-driven engineering, you can check out this book: Model-Driven Software Engineering in Practice (by Brambilla, Cabot and Wimmer).

To keep updated on my activities you can subscribe to the RSS feed of my blog or follow my twitter account (@MarcoBrambi).

IFML: opensource editor and university program

by Marco Brambilla, posted in data science

A bunch of new things are happening related to the recent OMG standard IFML (Interaction Flow Modeling Language):

1. A new opensource editor for the standard has been released on GitHub, together with appropriate introductory instructions and welcome page, thanks to the collaboration of WebRatio, Politecnico di Milano and Ecole des Mines de Nantes within the AutoMobile EU SME Research Project. The Editor has been developed as an EMF-based Eclipse plug-in using Obeo Sirius framework.

2. A new University Program has been released by WebRatio, which allows free access to the WebRatio Platform licenses, online multimedia and interactive learning materials, and free access to two WebRatio IFML Certification sessions.

3. And finally, stay tuned for the upcoming official delivery of Version 1.0 of the IFML standard, together with the new book on IFML, scheduled to be published shortly by Morgan Kauffman and the OMG Press (and already available for preview on Amazon!).
(and look at my tutorial on IFML to learn more on the language)
To keep updated on my activities you can subscribe to the RSS feed of my blog or follow my twitter account (@MarcoBrambi).