论文标题
倒数强化学习。如何从对抗性逆增强学习者中隐藏策略
Inverse-Inverse Reinforcement Learning. How to Hide Strategy from an Adversarial Inverse Reinforcement Learner
论文作者
论文摘要
逆增强学习（IRL）涉及从其行为中估算代理的效用功能。在本文中，我们考虑了代理如何隐藏其策略并减轻对抗性IRL攻击；我们称此为逆IRL（i-irl）。决策者应该如何选择其反应，以确保对对手进行IRL来估算代理商战略的对手重建其战略？本文包括四个结果：首先，我们提出了一种对抗性IRL算法，该算法在控制代理的实用程序函数时估算了代理的策略。我们的第二个结果i-rirl结果欺骗了对手使用的IRL算法。我们的ir-ir-Resust基于微观经济学的揭示偏好理论。关键的想法是，代理商要故意选择足够掩盖其真实策略的亚最佳响应。第三，当代理对对手指定的效用函数的嘈杂估计值时，我们为主I-Err结果给出了样本复杂性结果。最后，我们在雷达问题中说明了我们的I-rir方案，其中元认知雷达试图减轻对抗目标。
Inverse reinforcement learning (IRL) deals with estimating an agent's utility function from its actions. In this paper, we consider how an agent can hide its strategy and mitigate an adversarial IRL attack; we call this inverse IRL (I-IRL). How should the decision maker choose its response to ensure a poor reconstruction of its strategy by an adversary performing IRL to estimate the agent's strategy? This paper comprises four results: First, we present an adversarial IRL algorithm that estimates the agent's strategy while controlling the agent's utility function. Our second result for I-IRL result spoofs the IRL algorithm used by the adversary. Our I-IRL results are based on revealed preference theory in micro-economics. The key idea is for the agent to deliberately choose sub-optimal responses that sufficiently masks its true strategy. Third, we give a sample complexity result for our main I-IRL result when the agent has noisy estimates of the adversary specified utility function. Finally, we illustrate our I-IRL scheme in a radar problem where a meta-cognitive radar is trying to mitigate an adversarial target.
