Skip to main content

AS Seminar: User‘s feedback in Preference elicitation


The research studies optimal decision-making with the focus on preferences quantified for fully probabilistic design (FPD). FPD models the closed DM loop and the agent’s preferences by joint probability densities (pds). There is a preference-elicitation (PE) principle, which maps the agent’s model of the state transitions and its incompletely expressed wishes on an ideal pd quantifying them. This research also studies preferences targeting actions and contradicting preferences.
The gained algorithmic quantification provides ambitious but potentially reachable DM aims.
It suppresses demands on the agent selecting the preference-expressing inputs (a set of preferred states and preferred actions). The remaining PE options are: a parameter balancing exploration with exploitation; a fine specification of the ideal (desired) sets of states and actions; relative importance of these ideal sets.
In addition to the above, we add a meta-DM task to learn the user’s (agent’s) preferences and be able to find the free parameters optimally. The algorithmic “meta-agent” observes user’s satisfaction, expressed by school marks from 1 to 5 as in school and tunes the free PE inputs to improve these marks as much as possible. The solution requires a suitable formalisation of such a meta-task.
In my presentation, I will describe the principles of the meta-task and show experiments with feedback from different users and summarize improvements and shortcomings of the theory.

Submitted by neuner on