c9c0833 · opencog-rocca

examples/README.md +13 -322

…	…	@@ -2,330 +2,21 @@
2	2
3	3	Contains examples of OpencogAgent running in various environments.
4	4
5		## Chase
	5	* Chase: agent for a simplistic home-brewed gym env. Only 2 squares,
	6	with one pellet of food. The agent must chase and eat the food
	7	pellet to accumulate rewards.
6	8
7		### Description
	9	* CartPole: agent for the CartPole-v1 gym env.
8	10
9		Simplistic home-brewed gym env. 2 squares, with one pellet of food.
10		The agent must chase and eat the food pellet to accumulate rewards.
	11	* Pong: agent for Pong-v0 gym env.
11	12
12		### Status
	13	* Chase Malmo: Port of chase for Malmo. Food pellets are replaced by
	14	diamonds.
13	15
14		The agent is able to learn mono and poly action plans to chase and eat
15		the food pellet, either via purely direct evaluations (pattern
16		mining), or a combination of direct and indirect evaluations by
17		building more complex plans out of simpler plans using temporal
18		deduction. The optimal strategy is achieved with the following 2
19		plans:
	16	* Collect diamonds: agent for a simple minecraft environment which
	17	includes blocks of diamonds locked inside a house and a key to the
	18	house. The agent must look for the key, open the house and collect a
	19	diamond to get a reward.
20	20
21		1. `PelletPosition(None) ∧ AgentPosition(LeftSquare) ∧ do(GoRight) ⩘ do(Eat) ↝ Reward(1)`
22		2. `PelletPosition(None) ∧ AgentPosition(RightSquare) ∧ do(GoLeft) ⩘ do(Eat) ↝ Reward(1)`
23
24		represented in Atomese as
25
26		```scheme
27		(BackPredictiveImplicationScopeLink (stv 1 0.00669975)
28		(VariableSet
29		) ; [80075fffffff523a][3]
30		(SLink
31		(ZLink
32		) ; [800fbffffffe8ce4][3]
33		) ; [da5f815ba9d4009f][3]
34		(BackSequentialAndLink
35		(SLink
36		(ZLink
37		) ; [800fbffffffe8ce4][3]
38		) ; [da5f815ba9d4009f][3]
39		(AndLink (stv 0.03 0.2)
40		(EvaluationLink (stv 0.12 0.2)
41		(PredicateNode "Pellet Position") ; [56e6ab0f525cb504][3]
42		(ConceptNode "None") ; [68c616828fa0f8e6][3]
43		) ; [a609b2dbedb7904e][3]
44		(ExecutionLink
45		(SchemaNode "Go Right") ; [51c7a48fd94d12d8][3]
46		) ; [c29bf0559d1ad8ec][3]
47		(EvaluationLink (stv 0.37 0.2)
48		(PredicateNode "Agent Position") ; [3fdca752fd5e5335][3]
49		(ConceptNode "Left Square") ; [586f8a0db3b1388a][3]
50		) ; [ec8ec4a729ccab8c][3]
51		) ; [f07c6a1c8318095c][3]
52		(ExecutionLink
53		(SchemaNode "Eat") ; [3fe4e22345c3679f][3]
54		) ; [9efce1dc8918c209][3]
55		) ; [b7ee6db6cd3a43bf][3]
56		(EvaluationLink (stv 0.12 0.2)
57		(PredicateNode "Reward") ; [155bb4d713db0d51][3]
58		(NumberNode "1") ; [2cf0956d543cff8e][3]
59		) ; [d3cee8bdda06ffcb][3]
60		) ; [cac8b7b130c2115c][3]
61		```
62
63		```scheme
64		(BackPredictiveImplicationScopeLink (stv 1 0.00447761)
65		(VariableSet
66		) ; [80075fffffff523a][3]
67		(SLink
68		(ZLink
69		) ; [800fbffffffe8ce4][3]
70		) ; [da5f815ba9d4009f][3]
71		(BackSequentialAndLink
72		(SLink
73		(ZLink
74		) ; [800fbffffffe8ce4][3]
75		) ; [da5f815ba9d4009f][3]
76		(AndLink (stv 0.02 0.2)
77		(EvaluationLink (stv 0.12 0.2)
78		(PredicateNode "Pellet Position") ; [56e6ab0f525cb504][3]
79		(ConceptNode "None") ; [68c616828fa0f8e6][3]
80		) ; [a609b2dbedb7904e][3]
81		(ExecutionLink
82		(SchemaNode "Go Left") ; [7ca250f2efc2e872][3]
83		) ; [c7fb76d9605d5db5][3]
84		(EvaluationLink (stv 0.63 0.2)
85		(PredicateNode "Agent Position") ; [3fdca752fd5e5335][3]
86		(ConceptNode "Right Square") ; [6dd382acb6aa376e][3]
87		) ; [c9fcc2094e0150df][3]
88		) ; [e1b8c69b9a37aa32][3]
89		(ExecutionLink
90		(SchemaNode "Eat") ; [3fe4e22345c3679f][3]
91		) ; [9efce1dc8918c209][3]
92		) ; [b470646c0c97b387][3]
93		(EvaluationLink (stv 0.12 0.2)
94		(PredicateNode "Reward") ; [155bb4d713db0d51][3]
95		(NumberNode "1") ; [2cf0956d543cff8e][3]
96		) ; [d3cee8bdda06ffcb][3]
97		) ; [8f007faa792823de][3]
98		```
99
100		However, due to balancing exploration and exploitation, since ROCCA
101		also learns many more sub-optimal plans and has not full confidence
102		regarding the optimal plans (full confidence can only be
103		experientially obtained with infinitely many observations), it does
104		not achieve perfect score (which is 100). It however achieves greater
105		and greater scores after each learning session due to the confidences
106		of the optimal plans growing faster than that of the others.
107
108		### Usage
109
110		```bash
111		python chase.py
112		```
113
114		## CartPole
115
116		### Description
117
118		CartPole-v1 gym env.
119
120		### Status
121
122		No learning for now, the cognitive schemantics are hardcoded to move
123		the cart left or right when the pole deviates from the origin.
124
125		### Usage
126
127		```bash
128		python cartpole.py
129		```
130
131		## Pong
132
133		### Description
134
135		Pong-v0 gym env.
136
137		### Status
138
139		Unmantained. Would be nice to maintain it though, it is a simple yet
140		already challenging environment the delay between actions and rewards.
141		I.e. the agent needs to move the paddle early on in order to avoid
142		missing the ball later on.
143
144		### Requirements
145
146		You need to install the `atari` subpackage of gym, which you can do with
147
148		```
149		pip install gym[atari]
150		```
151
152		Additionally you need to install the Atari ROMS, download a ROM pack
153		from
154
155		http://www.atarimania.com/rom_collection_archive_atari_2600_roms.html
156
157		unrar it under the current folder
158
159		```
160		unrar e Roms.rar
161		```
162
163		It should create 2 zip files. Then you can install the ROMS with the
164		following command
165
166		```
167		python -m atari_py.import_roms <ROM_FOLDER>
168		```
169
170		You'll need to have `atari-py` properly installed for that.
171
172		### Usage
173
174		```bash
175		python pong.py
176		```
177
178		## Chase Malmo
179
180		### Description
181
182		Port of chase for Malmo. Food pellets are replaced by diamonds.
183
184		### Status
185
186		Unmaintained. As a close alternative, see the jupyter notebook file
187		`02_minerl_navigate_agent.ipynb` under the root folder which relies on
188		`minerl`.
189
190		### Usage
191
192		1. Clone malmo https://github.com/Microsoft/malmo
193		Note: No need to compile malmo as a pre-built version (MalmoPython) has been included
194		under `../rocca/malmo`. Only install the required dependencies to launch minecraft.
195
196		2. Launch `<MALMO>/Minecraft/launchClient.sh`.
197
198		3. Launch chase malmo `python chase_malmo.py`.
199
200
201		## Collect diamonds
202
203		### Description
204
205		A simple minecraft environment which includes blocks of diamonds locked
206		inside a house and a key to the house. The agent must look for
207		the key, open the house and collect a diamond to get a reward.
208
209		### Status
210
211		The Rocca agent is able to learn and plan the desired cognitive schematics
212		leading to getting/accumulating a reward via pattern mining and temporal deduction.
213
214		The three optimal action plans required to get a reward are:
215
216		1. `outside(self, house) ∧ do(go_to_key) ↝ hold(self, key)`
217
218		```scheme
219		(BackPredictiveImplicationScopeLink (stv 1 0.0111248)
220		(VariableSet
221		) ; [80075fffffff523a][1]
222		(SLink
223		(ZLink
224		) ; [800fbffffffe8ce4][1]
225		) ; [da5f815ba9d4009f][1]
226		(AndLink
227		(EvaluationLink (stv 0.64 0.0588235)
228		(PredicateNode "outside") ; [72730412e28a734][1]
229		(ListLink
230		(ConceptNode "self") ; [40b11d11524bd751][1]
231		(ConceptNode "house") ; [63eb9919f37daa5f][1]
232		) ; [aadca36fe9d1a468][1]
233		) ; [ca0c329fb1ab493b][1]
234		(ExecutionLink
235		(SchemaNode "go_to_key") ; [7f46c329a5e57604][1]
236		) ; [f8086e6fdf73cdf4][1]
237		) ; [ddda31153cf2aa6d][1]
238		(EvaluationLink (stv 0.32 0.0588235)
239		(PredicateNode "hold") ; [4b1b7a8b0a4d2853][1]
240		(ListLink
241		(ConceptNode "self") ; [40b11d11524bd751][1]
242		(ConceptNode "key") ; [4d0844146f96d3][1]
243		) ; [e7a9c95ae7484b28][1]
244		) ; [e6a8f21e6b37d8f0][1]
245		) ; [ff069058911f0233][1]
246		```
247		2. `outside(self, house) ∧ hold(self, key) ∧ do(go_to_house) ↝ inside(self, house)`
248
249		```scheme
250		(BackPredictiveImplicationScopeLink (stv 1 0.00621118)
251		(VariableSet
252		) ; [80075fffffff523a][1]
253		(SLink
254		(ZLink
255		) ; [800fbffffffe8ce4][1]
256		) ; [da5f815ba9d4009f][1]
257		(AndLink
258		(ExecutionLink
259		(SchemaNode "go_to_house") ; [7e1737e3e117d059][1]
260		) ; [93427319ec122fff][1]
261		(EvaluationLink (stv 0.64 0.0588235)
262		(PredicateNode "outside") ; [72730412e28a734][1]
263		(ListLink
264		(ConceptNode "self") ; [40b11d11524bd751][1]
265		(ConceptNode "house") ; [63eb9919f37daa5f][1]
266		) ; [aadca36fe9d1a468][1]
267		) ; [ca0c329fb1ab493b][1]
268		(EvaluationLink (stv 0.32 0.0588235)
269		(PredicateNode "hold") ; [4b1b7a8b0a4d2853][1]
270		(ListLink
271		(ConceptNode "self") ; [40b11d11524bd751][1]
272		(ConceptNode "key") ; [4d0844146f96d3][1]
273		) ; [e7a9c95ae7484b28][1]
274		) ; [e6a8f21e6b37d8f0][1]
275		) ; [fbf4892ec0643e2c][1]
276		(EvaluationLink
277		(PredicateNode "inside") ; [63398dcfcf85c8a3][1]
278		(ListLink
279		(ConceptNode "self") ; [40b11d11524bd751][1]
280		(ConceptNode "house") ; [63eb9919f37daa5f][1]
281		) ; [aadca36fe9d1a468][1]
282		) ; [871c182b52e89756][1]
283		) ; [a2984b519c1ddc4f][1]
284		```
285
286		3. `inside(self, house) ∧ do(go_to_diamonds) ↝ Reward(1)`
287
288		```scheme
289		(BackPredictiveImplicationScopeLink (stv 1 0.00621118)
290		(VariableSet
291		) ; [80075fffffff523a][1]
292		(SLink
293		(ZLink
294		) ; [800fbffffffe8ce4][1]
295		) ; [da5f815ba9d4009f][1]
296		(AndLink
297		(EvaluationLink
298		(PredicateNode "inside") ; [63398dcfcf85c8a3][1]
299		(ListLink
300		(ConceptNode "self") ; [40b11d11524bd751][1]
301		(ConceptNode "house") ; [63eb9919f37daa5f][1]
302		) ; [aadca36fe9d1a468][1]
303		) ; [871c182b52e89756][1]
304		(ExecutionLink
305		(SchemaNode "go_to_diamonds") ; [7aee74cf6bad6442][1]
306		) ; [a2630f96ffbe0861][1]
307		) ; [b4373ec3773f1783][1]
308		(EvaluationLink
309		(PredicateNode "Reward") ; [155bb4d713db0d51][1]
310		(NumberNode "1") ; [2cf0956d543cff8e][1]
311		) ; [d3cee8bdda06ffcb][1]
312		) ; [8a298365b46b204b][1]
313		```
314
315		### Usage
316
317		1. Clone malmo https://github.com/Microsoft/malmo
318		Note: No need to compile malmo as a pre-built version (MalmoPython) has been included
319		under `../rocca/malmo`. Only install the required dependencies to launch minecraft.
320
321		2. Launch `<MALMO>/Minecraft/launchClient.sh`.
322
323		3. Launch collect diamonds `python collect_diamonds.py`.
324
325		- Note: If there is a need to visualize mined cognitive schematics, do the following before step 3.
326
327		- Build and run atompace-explorer here https://github.com/tanksha/atomspace-explorer rocca_demo branch.
328
329		- Start the RESTAPI `python start_rest_service.py`
330
331		- Open `collect_diamonds.py` and set `self.visualize_cogscm = True`
	21	Browse the various subfolders and see their README.md files for more
	22	information.

examples/__init__.py +0 -0

(no content changes — binary or rename)

examples/cartpole.py +0 -68

…	…	@@ -1,68 +0,0 @@
1		# Define an agent for the CartPole env with GymAgent
2
3		##############
4		# Initialize #
5		##############
6
7		# Python
8		import time
9		from typing import List
10
11		# OpenCog
12		from opencog.atomspace import AtomSpace
13		from opencog.utilities import set_default_atomspace
14		from opencog.type_constructors import *
15		from opencog.spacetime import *
16		from opencog.pln import *
17		from opencog.ure import ure_logger
18		from opencog.logger import log
19
20		# OpenAI Gym
21		import gym
22
23		env = gym.make("CartPole-v1")
24		# Uncomment the following to get a description of env
25		# help(env.unwrapped)
26
27		# OpenCog Gym
28		from rocca.agents.cartpole import FixedCartPoleAgent
29		from rocca.agents.utils import *
30		from rocca.envs.wrappers import GymWrapper, CartPoleWrapper
31
32		from rocca.utils import *
33
34
35		########
36		# Main #
37		########
38		def main():
39		# Init loggers
40		log.set_level("fine")
41		log.set_sync(False)
42		agent_log.set_level("fine")
43		agent_log.set_sync(False)
44		ure_logger().set_level("fine")
45		ure_logger().set_sync(False)
46
47		# Set main atomspace
48		atomspace = AtomSpace()
49		set_default_atomspace(atomspace)
50
51		# Wrap environment
52		wrapped_env = CartPoleWrapper(env, atomspace)
53
54		# Instantiate CartPoleAgent, and tune parameters
55		cpa = FixedCartPoleAgent(wrapped_env, atomspace)
56		cpa.delta = 1.0e-16
57
58		# Run control loop
59		while not cpa.control_cycle():
60		wrapped_env.render()
61		time.sleep(0.1)
62		log.info("cycle_count = {}".format(cpa.cycle_count))
63
64		log_msg(agent_log, f"The final reward is {cpa.accumulated_reward}.")
65
66
67		if __name__ == "__main__":
68		main()

examples/cartpole/README.md +25 -0

…	…	@@ -0,0 +1,25 @@
	1	# CartPole
	2
	3	## Description
	4
	5	Agent for the CartPole-v1 gym env.
	6
	7	## Status
	8
	9	No learning for now, the cognitive schemantics are hardcoded to move
	10	the cart left or right when the pole deviates from the origin.
	11
	12	## Usage
	13
	14	```bash
	15	python cartpole.py
	16	```
	17
	18	## Jupiter Notebook
	19
	20	In addition, there is a Jupyter notebook under the project folder,
	21	which can be called as follows
	22
	23	```bash
	24	jupyter notebook ../../01_cartpole.ipynb
	25	```

examples/cartpole/cartpole.py +68 -0

…	…	@@ -0,0 +1,68 @@
	1	# Define an agent for the CartPole env with GymAgent
	2
	3	##############
	4	# Initialize #
	5	##############
	6
	7	# Python
	8	import time
	9	from typing import List
	10
	11	# OpenCog
	12	from opencog.atomspace import AtomSpace
	13	from opencog.utilities import set_default_atomspace
	14	from opencog.type_constructors import *
	15	from opencog.spacetime import *
	16	from opencog.pln import *
	17	from opencog.ure import ure_logger
	18	from opencog.logger import log
	19
	20	# OpenAI Gym
	21	import gym
	22
	23	env = gym.make("CartPole-v1")
	24	# Uncomment the following to get a description of env
	25	# help(env.unwrapped)
	26
	27	# OpenCog Gym
	28	from rocca.agents.cartpole import FixedCartPoleAgent
	29	from rocca.agents.utils import *
	30	from rocca.envs.wrappers import GymWrapper, CartPoleWrapper
	31
	32	from rocca.utils import *
	33
	34
	35	########
	36	# Main #
	37	########
	38	def main():
	39	# Init loggers
	40	log.set_level("fine")
	41	log.set_sync(False)
	42	agent_log.set_level("fine")
	43	agent_log.set_sync(False)
	44	ure_logger().set_level("fine")
	45	ure_logger().set_sync(False)
	46
	47	# Set main atomspace
	48	atomspace = AtomSpace()
	49	set_default_atomspace(atomspace)
	50
	51	# Wrap environment
	52	wrapped_env = CartPoleWrapper(env, atomspace)
	53
	54	# Instantiate CartPoleAgent, and tune parameters
	55	cpa = FixedCartPoleAgent(wrapped_env, atomspace)
	56	cpa.delta = 1.0e-16
	57
	58	# Run control loop
	59	while not cpa.control_cycle():
	60	wrapped_env.render()
	61	time.sleep(0.1)
	62	log.info("cycle_count = {}".format(cpa.cycle_count))
	63
	64	log_msg(agent_log, f"The final reward is {cpa.accumulated_reward}.")
	65
	66
	67	if __name__ == "__main__":
	68	main()

examples/chase.py +0 -191

…	…	@@ -1,191 +0,0 @@
1		# Define an agent for the Chase env with OpencogAgent
2
3		##############
4		# Initialize #
5		##############
6
7		# Python
8		import time
9
10		# OpenAI Gym
11		import gym
12
13		# OpenCog
14		from opencog.type_constructors import (
15		ExecutionLink,
16		SchemaNode,
17		EvaluationLink,
18		PredicateNode,
19		NumberNode,
20		ConceptNode,
21		)
22		from opencog.utilities import set_default_atomspace
23		from opencog.pln import *
24		from opencog.ure import ure_logger
25		from opencog.logger import log
26
27		# ROCCA
28		from rocca.agents import OpencogAgent
29		from rocca.agents.utils import *
30		from rocca.envs.wrappers import GymWrapper
31
32		env = gym.make("Chase-v0")
33		# Uncomment the following to get a description of env
34		# help(env.unwrapped)
35
36		#################
37		# Chase Wrapper #
38		#################
39		# TODO: move to a library module.
40		class ChaseWrapper(GymWrapper):
41		def __init__(self, env, atomspace):
42		action_list = ["Go Left", "Go Right", "Stay", "Eat"]
43		super().__init__(env, atomspace, action_list)
44
45		def labeled_observation(self, space, obs, sbs=""):
46		"""Translate gym observation to Atomese
47
48		There are 2 gym observations:
49
50		Agent Position is 0 (left) or 1 (right)
51		Pellet Positon is 0 (left), 1 (right) or 2 (none)
52
53		Translated in Atomese as follows:
54
55		Evaluation
56		Predicate "Agent Position"
57		AP
58
59		where AP can be
60
61		1. Concept "Left Square"
62		2. Concept "Right Square"
63
64		Evaluation
65		Predicate "Pellet Position"
66		PP
67
68		where PP can be
69
70		1. Concept "Left Square"
71		2. Concept "Right Square"
72		3. Concept "None"
73
74		"""
75
76		to_atomese_position = {
77		0: ConceptNode("Left Square"),
78		1: ConceptNode("Right Square"),
79		2: ConceptNode("None"),
80		}
81		ap = to_atomese_position[obs[0]]
82		pp = to_atomese_position[obs[1]]
83		return [
84		EvaluationLink(PredicateNode("Agent Position"), ap),
85		EvaluationLink(PredicateNode("Pellet Position"), pp),
86		]
87
88
89		###############
90		# Chase Agent #
91		###############
92
93
94		class ChaseAgent(OpencogAgent):
95		def __init__(self, env, atomspace):
96		set_default_atomspace(atomspace)
97
98		# Create Action Space. The set of allowed actions an agent can take.
99		# TODO take care of action parameters.
100		action_space = {ExecutionLink(SchemaNode(a)) for a in env.action_names}
101
102		# Create Goal
103		pgoal = EvaluationLink(PredicateNode("Reward"), NumberNode("1"))
104		ngoal = EvaluationLink(PredicateNode("Reward"), NumberNode("0"))
105
106		# Call super ctor
107		OpencogAgent.__init__(self, env, atomspace, action_space, pgoal, ngoal)
108
109		# Overwrite some OpencogAgent parameters
110		self.polyaction_mining = False
111		self.monoaction_general_succeedent_mining = True
112		self.temporal_deduction = True
113		self.cogscm_minimum_strength = 0.9
114		self.cogscm_maximum_shannon_entropy = 1
115		self.cogscm_maximum_differential_entropy = 0
116		self.cogscm_maximum_variables = 0
117
118
119		if __name__ == "__main__":
120		# Set main atomspace
121		atomspace = AtomSpace()
122		set_default_atomspace(atomspace)
123
124		# Init loggers
125		log.set_level("info")
126		# log.set_sync(True)
127		agent_log.set_level("info")
128		# agent_log.set_sync(True)
129		ure_logger().set_level("info")
130		# ure_logger().set_sync(True)
131		miner_log = MinerLogger(atomspace)
132		miner_log.set_level("info")
133		# miner_log.set_sync(True)
134
135		# Wrap environment
136		wrapped_env = ChaseWrapper(env, atomspace)
137
138		# ChaseAgent
139		cag = ChaseAgent(wrapped_env, atomspace)
140
141		# Log all parameters of cag, useful for debugging
142		cag.log_parameters(level="debug")
143
144		# Uncomment to load previously saved percepta or cogscms
145		# atomspaces.
146		# cag.load_cogscms_atomspace("chase_cogscms_atomspace.scm")
147		# cag.load_percepta_atomspace("chase_percepta_atomspace.scm")
148
149		# Training/learning loop
150		lt_iterations = 2 # Number of learning-training iterations
151		lt_period = 200 # Duration of a learning-training iteration
152		max_cycle_count = cag.cycle_count + lt_iterations * lt_period
153		for i in range(lt_iterations):
154		wrapped_env.restart()
155		cag.reset_action_counter()
156		par = cag.accumulated_reward # Keep track of the reward before
157
158		# Discover patterns to make more informed decisions
159		agent_log.info("Start learning ({}/{})".format(i + 1, lt_iterations))
160		cag.learn()
161
162		# Run agent to accumulate percepta
163		agent_log.info("Start training ({}/{})".format(i + 1, lt_iterations))
164		for j in range(lt_period):
165		cag.control_cycle()
166		wrapped_env.render()
167		time.sleep(0.1)
168		agent_log.info(
169		"Control cycle ({}/{})".format(
170		cag.cycle_count, max_cycle_count
171		)
172		)
173		nar = cag.accumulated_reward - par
174		agent_log.info(
175		"Accumulated reward during {}th iteration = {}".format(i + 1, nar)
176		)
177		agent_log.info(
178		"Action counter during {}th iteration:\n{}".format(
179		i + 1, cag.action_counter_to_str()
180		)
181		)
182
183		# Log all agent's atomspaces at the end (at fine level)
184		agent_log_atomspace(cag.atomspace, "fine", "cag.atomspace")
185		agent_log_atomspace(cag.percepta_atomspace, "fine", "cag.percepta_atomspace")
186		agent_log_atomspace(cag.cogscms_atomspace, "fine", "cag.cogscms_atomspace")
187		agent_log_atomspace(cag.working_atomspace, "fine", "cag.working_atomspace")
188
189		# Uncomment to save percepta or cogscms atomspaces
190		# cag.save_percepta_atomspace("chase_percepta_atomspace.scm")
191		# cag.save_cogscms_atomspace("chase_cogscms_atomspace.scm")

examples/chase/README.md +109 -0

…	…	@@ -0,0 +1,109 @@
	1	# Chase
	2
	3	## Description
	4
	5	Agent for a simplistic home-brewed gym env. 2 squares, with one
	6	pellet of food. The agent must chase and eat the food pellet to
	7	accumulate rewards.
	8
	9	## Status
	10
	11	The agent is able to learn mono and poly action plans to chase and eat
	12	the food pellet, either via purely direct evaluations (pattern
	13	mining), or a combination of direct and indirect evaluations by
	14	building more complex plans out of simpler plans using temporal
	15	deduction. The optimal strategy is achieved with the following 2
	16	plans:
	17
	18	1. `PelletPosition(None) ∧ AgentPosition(LeftSquare) ∧ do(GoRight) ⩘ do(Eat) ↝ Reward(1)`
	19	2. `PelletPosition(None) ∧ AgentPosition(RightSquare) ∧ do(GoLeft) ⩘ do(Eat) ↝ Reward(1)`
	20
	21	represented in Atomese as
	22
	23	```scheme
	24	(BackPredictiveImplicationScopeLink (stv 1 0.00669975)
	25	(VariableSet
	26	) ; [80075fffffff523a][3]
	27	(SLink
	28	(ZLink
	29	) ; [800fbffffffe8ce4][3]
	30	) ; [da5f815ba9d4009f][3]
	31	(BackSequentialAndLink
	32	(SLink
	33	(ZLink
	34	) ; [800fbffffffe8ce4][3]
	35	) ; [da5f815ba9d4009f][3]
	36	(AndLink (stv 0.03 0.2)
	37	(EvaluationLink (stv 0.12 0.2)
	38	(PredicateNode "Pellet Position") ; [56e6ab0f525cb504][3]
	39	(ConceptNode "None") ; [68c616828fa0f8e6][3]
	40	) ; [a609b2dbedb7904e][3]
	41	(ExecutionLink
	42	(SchemaNode "Go Right") ; [51c7a48fd94d12d8][3]
	43	) ; [c29bf0559d1ad8ec][3]
	44	(EvaluationLink (stv 0.37 0.2)
	45	(PredicateNode "Agent Position") ; [3fdca752fd5e5335][3]
	46	(ConceptNode "Left Square") ; [586f8a0db3b1388a][3]
	47	) ; [ec8ec4a729ccab8c][3]
	48	) ; [f07c6a1c8318095c][3]
	49	(ExecutionLink
	50	(SchemaNode "Eat") ; [3fe4e22345c3679f][3]
	51	) ; [9efce1dc8918c209][3]
	52	) ; [b7ee6db6cd3a43bf][3]
	53	(EvaluationLink (stv 0.12 0.2)
	54	(PredicateNode "Reward") ; [155bb4d713db0d51][3]
	55	(NumberNode "1") ; [2cf0956d543cff8e][3]
	56	) ; [d3cee8bdda06ffcb][3]
	57	) ; [cac8b7b130c2115c][3]
	58	```
	59
	60	```scheme
	61	(BackPredictiveImplicationScopeLink (stv 1 0.00447761)
	62	(VariableSet
	63	) ; [80075fffffff523a][3]
	64	(SLink
	65	(ZLink
	66	) ; [800fbffffffe8ce4][3]
	67	) ; [da5f815ba9d4009f][3]
	68	(BackSequentialAndLink
	69	(SLink
	70	(ZLink
	71	) ; [800fbffffffe8ce4][3]
	72	) ; [da5f815ba9d4009f][3]
	73	(AndLink (stv 0.02 0.2)
	74	(EvaluationLink (stv 0.12 0.2)
	75	(PredicateNode "Pellet Position") ; [56e6ab0f525cb504][3]
	76	(ConceptNode "None") ; [68c616828fa0f8e6][3]
	77	) ; [a609b2dbedb7904e][3]
	78	(ExecutionLink
	79	(SchemaNode "Go Left") ; [7ca250f2efc2e872][3]
	80	) ; [c7fb76d9605d5db5][3]
	81	(EvaluationLink (stv 0.63 0.2)
	82	(PredicateNode "Agent Position") ; [3fdca752fd5e5335][3]
	83	(ConceptNode "Right Square") ; [6dd382acb6aa376e][3]
	84	) ; [c9fcc2094e0150df][3]
	85	) ; [e1b8c69b9a37aa32][3]
	86	(ExecutionLink
	87	(SchemaNode "Eat") ; [3fe4e22345c3679f][3]
	88	) ; [9efce1dc8918c209][3]
	89	) ; [b470646c0c97b387][3]
	90	(EvaluationLink (stv 0.12 0.2)
	91	(PredicateNode "Reward") ; [155bb4d713db0d51][3]
	92	(NumberNode "1") ; [2cf0956d543cff8e][3]
	93	) ; [d3cee8bdda06ffcb][3]
	94	) ; [8f007faa792823de][3]
	95	```
	96
	97	However, due to balancing exploration and exploitation, since ROCCA
	98	also learns many more sub-optimal plans and has not full confidence
	99	regarding the optimal plans (full confidence can only be
	100	experientially obtained with infinitely many observations), it does
	101	not achieve perfect score (which is 100). It however achieves greater
	102	and greater scores after each learning session due to the confidences
	103	of the optimal plans growing faster than that of the others.
	104
	105	## Usage
	106
	107	```bash
	108	python chase.py
	109	```

examples/chase/chase.py +191 -0

…	…	@@ -0,0 +1,191 @@
	1	# Define an agent for the Chase env with OpencogAgent
	2
	3	##############
	4	# Initialize #
	5	##############
	6
	7	# Python
	8	import time
	9
	10	# OpenAI Gym
	11	import gym
	12
	13	# OpenCog
	14	from opencog.type_constructors import (
	15	ExecutionLink,
	16	SchemaNode,
	17	EvaluationLink,
	18	PredicateNode,
	19	NumberNode,
	20	ConceptNode,
	21	)
	22	from opencog.utilities import set_default_atomspace
	23	from opencog.pln import *
	24	from opencog.ure import ure_logger
	25	from opencog.logger import log
	26
	27	# ROCCA
	28	from rocca.agents import OpencogAgent
	29	from rocca.agents.utils import *
	30	from rocca.envs.wrappers import GymWrapper
	31
	32	env = gym.make("Chase-v0")
	33	# Uncomment the following to get a description of env
	34	# help(env.unwrapped)
	35
	36	#################
	37	# Chase Wrapper #
	38	#################
	39	# TODO: move to a library module.
	40	class ChaseWrapper(GymWrapper):
	41	def __init__(self, env, atomspace):
	42	action_list = ["Go Left", "Go Right", "Stay", "Eat"]
	43	super().__init__(env, atomspace, action_list)
	44
	45	def labeled_observation(self, space, obs, sbs=""):
	46	"""Translate gym observation to Atomese
	47
	48	There are 2 gym observations:
	49
	50	Agent Position is 0 (left) or 1 (right)
	51	Pellet Positon is 0 (left), 1 (right) or 2 (none)
	52
	53	Translated in Atomese as follows:
	54
	55	Evaluation
	56	Predicate "Agent Position"
	57	AP
	58
	59	where AP can be
	60
	61	1. Concept "Left Square"
	62	2. Concept "Right Square"
	63
	64	Evaluation
	65	Predicate "Pellet Position"
	66	PP
	67
	68	where PP can be
	69
	70	1. Concept "Left Square"
	71	2. Concept "Right Square"
	72	3. Concept "None"
	73
	74	"""
	75
	76	to_atomese_position = {
	77	0: ConceptNode("Left Square"),
	78	1: ConceptNode("Right Square"),
	79	2: ConceptNode("None"),
	80	}
	81	ap = to_atomese_position[obs[0]]
	82	pp = to_atomese_position[obs[1]]
	83	return [
	84	EvaluationLink(PredicateNode("Agent Position"), ap),
	85	EvaluationLink(PredicateNode("Pellet Position"), pp),
	86	]
	87
	88
	89	###############
	90	# Chase Agent #
	91	###############
	92
	93
	94	class ChaseAgent(OpencogAgent):
	95	def __init__(self, env, atomspace):
	96	set_default_atomspace(atomspace)
	97
	98	# Create Action Space. The set of allowed actions an agent can take.
	99	# TODO take care of action parameters.
	100	action_space = {ExecutionLink(SchemaNode(a)) for a in env.action_names}
	101
	102	# Create Goal
	103	pgoal = EvaluationLink(PredicateNode("Reward"), NumberNode("1"))
	104	ngoal = EvaluationLink(PredicateNode("Reward"), NumberNode("0"))
	105
	106	# Call super ctor
	107	OpencogAgent.__init__(self, env, atomspace, action_space, pgoal, ngoal)
	108
	109	# Overwrite some OpencogAgent parameters
	110	self.polyaction_mining = False
	111	self.monoaction_general_succeedent_mining = True
	112	self.temporal_deduction = True
	113	self.cogscm_minimum_strength = 0.9
	114	self.cogscm_maximum_shannon_entropy = 1
	115	self.cogscm_maximum_differential_entropy = 0
	116	self.cogscm_maximum_variables = 0
	117
	118
	119	if __name__ == "__main__":
	120	# Set main atomspace
	121	atomspace = AtomSpace()
	122	set_default_atomspace(atomspace)
	123
	124	# Init loggers
	125	log.set_level("info")
	126	# log.set_sync(True)
	127	agent_log.set_level("info")
	128	# agent_log.set_sync(True)
	129	ure_logger().set_level("info")
	130	# ure_logger().set_sync(True)
	131	miner_log = MinerLogger(atomspace)
	132	miner_log.set_level("info")
	133	# miner_log.set_sync(True)
	134
	135	# Wrap environment
	136	wrapped_env = ChaseWrapper(env, atomspace)
	137
	138	# ChaseAgent
	139	cag = ChaseAgent(wrapped_env, atomspace)
	140
	141	# Log all parameters of cag, useful for debugging
	142	cag.log_parameters(level="debug")
	143
	144	# Uncomment to load previously saved percepta or cogscms
	145	# atomspaces.
	146	# cag.load_cogscms_atomspace("chase_cogscms_atomspace.scm")
	147	# cag.load_percepta_atomspace("chase_percepta_atomspace.scm")
	148
	149	# Training/learning loop
	150	lt_iterations = 2 # Number of learning-training iterations
	151	lt_period = 200 # Duration of a learning-training iteration
	152	max_cycle_count = cag.cycle_count + lt_iterations * lt_period
	153	for i in range(lt_iterations):
	154	wrapped_env.restart()
	155	cag.reset_action_counter()
	156	par = cag.accumulated_reward # Keep track of the reward before
	157
	158	# Discover patterns to make more informed decisions
	159	agent_log.info("Start learning ({}/{})".format(i + 1, lt_iterations))
	160	cag.learn()
	161
	162	# Run agent to accumulate percepta
	163	agent_log.info("Start training ({}/{})".format(i + 1, lt_iterations))
	164	for j in range(lt_period):
	165	cag.control_cycle()
	166	wrapped_env.render()
	167	time.sleep(0.1)
	168	agent_log.info(
	169	"Control cycle ({}/{})".format(
	170	cag.cycle_count, max_cycle_count
	171	)
	172	)
	173	nar = cag.accumulated_reward - par
	174	agent_log.info(
	175	"Accumulated reward during {}th iteration = {}".format(i + 1, nar)
	176	)
	177	agent_log.info(
	178	"Action counter during {}th iteration:\n{}".format(
	179	i + 1, cag.action_counter_to_str()
	180	)
	181	)
	182
	183	# Log all agent's atomspaces at the end (at fine level)
	184	agent_log_atomspace(cag.atomspace, "fine", "cag.atomspace")
	185	agent_log_atomspace(cag.percepta_atomspace, "fine", "cag.percepta_atomspace")
	186	agent_log_atomspace(cag.cogscms_atomspace, "fine", "cag.cogscms_atomspace")
	187	agent_log_atomspace(cag.working_atomspace, "fine", "cag.working_atomspace")
	188
	189	# Uncomment to save percepta or cogscms atomspaces
	190	# cag.save_percepta_atomspace("chase_percepta_atomspace.scm")
	191	# cag.save_cogscms_atomspace("chase_cogscms_atomspace.scm")

examples/chase_malmo.py +0 -109

…	…	@@ -1,109 +0,0 @@
1		# Define an agent for the Chase env with OpencogAgent
2		#
3		# WARNING: unmaintained
4
5		##############
6		# Initialize #
7		##############
8
9		# Python
10		import time
11
12		# OpenCog
13		from opencog.logger import log
14		from opencog.pln import *
15		from opencog.type_constructors import *
16		from opencog.utilities import set_default_atomspace
17
18		from rocca.agents import OpencogAgent
19		from rocca.agents.utils import agent_log
20		from rocca.envs.malmo_demo.chase_env import mission_xml
21		from rocca.envs.wrappers import MalmoWrapper
22		from opencog.ure import ure_logger
23
24		###############
25		# Chase Agent #
26		###############
27		from rocca.envs.wrappers.utils import mk_action
28
29
30		class ChaseAgent(OpencogAgent):
31		def __init__(self, env, atomspace, action_space, p_goal, n_goal):
32		OpencogAgent.__init__(self, env, atomspace, action_space, p_goal, n_goal)
33		env.step(mk_action("hotbar.9", 1)) # Press the hotbar key
34		env.step(
35		mk_action("hotbar.9", 0)
36		) # Release hotbar key - agent should now be holding diamond_pickaxe
37
38		def eat(self, i):
39		self.env.step(mk_action("move", 0))
40		self.env.step(mk_action("hotbar.{}".format(i), 1))
41		self.env.step(mk_action("hotbar.{}".format(i), 0))
42		self.env.step(mk_action("use", 1))
43
44		def wake(self):
45		self.env.step(mk_action("use", 0))
46		self.env.step(mk_action("hotbar.9", 1))
47		self.env.step(mk_action("hotbar.9", 0))
48
49
50		if __name__ == "__main__":
51		atomspace = AtomSpace()
52		set_default_atomspace(atomspace)
53
54		# Init loggers
55		log.set_level("info")
56		# log.set_sync(True)
57		agent_log.set_level("debug")
58		# agent_log.set_sync(True)
59		ure_logger().set_level("debug")
60
61		# Wrap environment
62		wrapped_env = MalmoWrapper(missionXML=mission_xml, validate=True)
63
64		# Create Goal
65		pgoal = EvaluationLink(PredicateNode("Reward"), NumberNode("1"))
66		ngoal = EvaluationLink(PredicateNode("Reward"), NumberNode("0"))
67
68		# Create Action Space. The set of allowed actions an agent can take.
69		# TODO take care of action parameters.
70		action_space = {
71		ExecutionLink(SchemaNode("tpz"), NumberNode("2.5")),
72		ExecutionLink(SchemaNode("tpz"), NumberNode("-1.5")),
73		ExecutionLink(SchemaNode("attack"), NumberNode("0")),
74		ExecutionLink(SchemaNode("attack"), NumberNode("1")),
75		ExecutionLink(SchemaNode("move"), NumberNode("0")),
76		ExecutionLink(SchemaNode("move"), NumberNode("0.5")),
77		}
78
79		# ChaseAgent
80		ca = ChaseAgent(wrapped_env, atomspace, action_space, pgoal, ngoal)
81
82		# Eat some food.
83		ca.eat(4)
84		time.sleep(5)
85
86		# Training/learning loop
87		lt_iterations = 3 # Number of learning-training iterations
88		lt_period = 200 # Duration of a learning-training iteration
89		for i in range(lt_iterations):
90		ca.reset_action_counter()
91		par = ca.accumulated_reward # Keep track of the reward before
92		# Discover patterns to make more informed decisions
93		agent_log.info("Start learning ({}/{})".format(i + 1, lt_iterations))
94		ca.learn()
95		ca.wake()
96		# Run agent to accumulate percepta
97		agent_log.info("Start training ({}/{})".format(i + 1, lt_iterations))
98		for j in range(lt_period):
99		ca.control_cycle()
100		time.sleep(0.01)
101		log.info("cycle_count = {}".format(ca.cycle_count))
102		nar = ca.accumulated_reward - par
103		agent_log.info(
104		"Accumulated reward during {}th iteration = {}".format(i + 1, nar)
105		)
106		agent_log.info(
107		"Action counter during {}th iteration:\n{}".format(i + 1, ca.action_counter)
108		)
109		ca.eat(8 - i)

examples/chase_malmo/README.md +23 -0

…	…	@@ -0,0 +1,23 @@
	1	# Chase Malmo
	2
	3	## Description
	4
	5	Port of chase for Malmo. Food pellets are replaced by diamonds.
	6
	7	## Status
	8
	9	It should work. Though as a close alternative, you may find a jupyter
	10	notebook file `02_minerl_navigate_agent.ipynb` under the project
	11	folder which relies on `minerl`.
	12
	13	## Usage
	14
	15	1. Clone malmo https://github.com/Microsoft/malmo
	16
	17	Note: No need to compile malmo as a pre-built version (MalmoPython)
	18	has been included under `../../rocca/malmo`. Only install the required
	19	dependencies to launch minecraft.
	20
	21	2. Launch `<MALMO>/Minecraft/launchClient.sh`.
	22
	23	3. Launch chase malmo `python chase_malmo.py`.

examples/chase_malmo/chase_malmo.py +109 -0

…	…	@@ -0,0 +1,109 @@
	1	# Define an agent for the Chase env with OpencogAgent
	2	#
	3	# WARNING: unmaintained
	4
	5	##############
	6	# Initialize #
	7	##############
	8
	9	# Python
	10	import time
	11
	12	# OpenCog
	13	from opencog.logger import log
	14	from opencog.pln import *
	15	from opencog.type_constructors import *
	16	from opencog.utilities import set_default_atomspace
	17
	18	from rocca.agents import OpencogAgent
	19	from rocca.agents.utils import agent_log
	20	from rocca.envs.malmo_demo.chase_env import mission_xml
	21	from rocca.envs.wrappers import MalmoWrapper
	22	from opencog.ure import ure_logger
	23
	24	###############
	25	# Chase Agent #
	26	###############
	27	from rocca.envs.wrappers.utils import mk_action
	28
	29
	30	class ChaseAgent(OpencogAgent):
	31	def __init__(self, env, atomspace, action_space, p_goal, n_goal):
	32	OpencogAgent.__init__(self, env, atomspace, action_space, p_goal, n_goal)
	33	env.step(mk_action("hotbar.9", 1)) # Press the hotbar key
	34	env.step(
	35	mk_action("hotbar.9", 0)
	36	) # Release hotbar key - agent should now be holding diamond_pickaxe
	37
	38	def eat(self, i):
	39	self.env.step(mk_action("move", 0))
	40	self.env.step(mk_action("hotbar.{}".format(i), 1))
	41	self.env.step(mk_action("hotbar.{}".format(i), 0))
	42	self.env.step(mk_action("use", 1))
	43
	44	def wake(self):
	45	self.env.step(mk_action("use", 0))
	46	self.env.step(mk_action("hotbar.9", 1))
	47	self.env.step(mk_action("hotbar.9", 0))
	48
	49
	50	if __name__ == "__main__":
	51	atomspace = AtomSpace()
	52	set_default_atomspace(atomspace)
	53
	54	# Init loggers
	55	log.set_level("info")
	56	# log.set_sync(True)
	57	agent_log.set_level("debug")
	58	# agent_log.set_sync(True)
	59	ure_logger().set_level("debug")
	60
	61	# Wrap environment
	62	wrapped_env = MalmoWrapper(missionXML=mission_xml, validate=True)
	63
	64	# Create Goal
	65	pgoal = EvaluationLink(PredicateNode("Reward"), NumberNode("1"))
	66	ngoal = EvaluationLink(PredicateNode("Reward"), NumberNode("0"))
	67
	68	# Create Action Space. The set of allowed actions an agent can take.
	69	# TODO take care of action parameters.
	70	action_space = {
	71	ExecutionLink(SchemaNode("tpz"), NumberNode("2.5")),
	72	ExecutionLink(SchemaNode("tpz"), NumberNode("-1.5")),
	73	ExecutionLink(SchemaNode("attack"), NumberNode("0")),
	74	ExecutionLink(SchemaNode("attack"), NumberNode("1")),
	75	ExecutionLink(SchemaNode("move"), NumberNode("0")),
	76	ExecutionLink(SchemaNode("move"), NumberNode("0.5")),
	77	}
	78
	79	# ChaseAgent
	80	ca = ChaseAgent(wrapped_env, atomspace, action_space, pgoal, ngoal)
	81
	82	# Eat some food.
	83	ca.eat(4)
	84	time.sleep(5)
	85
	86	# Training/learning loop
	87	lt_iterations = 3 # Number of learning-training iterations
	88	lt_period = 200 # Duration of a learning-training iteration
	89	for i in range(lt_iterations):
	90	ca.reset_action_counter()
	91	par = ca.accumulated_reward # Keep track of the reward before
	92	# Discover patterns to make more informed decisions
	93	agent_log.info("Start learning ({}/{})".format(i + 1, lt_iterations))
	94	ca.learn()
	95	ca.wake()
	96	# Run agent to accumulate percepta
	97	agent_log.info("Start training ({}/{})".format(i + 1, lt_iterations))
	98	for j in range(lt_period):
	99	ca.control_cycle()
	100	time.sleep(0.01)
	101	log.info("cycle_count = {}".format(ca.cycle_count))
	102	nar = ca.accumulated_reward - par
	103	agent_log.info(
	104	"Accumulated reward during {}th iteration = {}".format(i + 1, nar)
	105	)
	106	agent_log.info(
	107	"Action counter during {}th iteration:\n{}".format(i + 1, ca.action_counter)
	108	)
	109	ca.eat(8 - i)

examples/collect_diamonds.py +0 -192

…	…	@@ -1,192 +0,0 @@
1		import time
2
3		# OpenCog
4		from opencog.logger import log
5		from opencog.pln import *
6		from opencog.type_constructors import *
7		from opencog.utilities import set_default_atomspace
8		from opencog.ure import ure_logger
9
10
11		from rocca.agents import OpencogAgent
12		from rocca.agents.utils import TRUE_TV, agent_log
13		from rocca.envs.malmo_demo.collect_diamonds_env import *
14		from rocca.envs.wrappers import MalmoWrapper
15		from rocca.envs.wrappers.utils import mk_evaluation
16		from rocca.agents.utils import *
17		from functools import wraps
18		import os
19		import requests
20
21
22		class AbstractMalmoWrapper(MalmoWrapper):
23		def __init__(self, missionXML, validate):
24		MalmoWrapper.__init__(self, missionXML, validate)
25
26		@staticmethod
27		def step_decorator(step):
28		@wraps(step)
29		def wrapper(ref, action, update_callback=None):
30		name = action.out[0]
31		return step(ref, name, update_callback)
32
33		return wrapper
34
35		@step_decorator.__func__
36		def step(self, abstract_action, update_callback=None):
37		try:
38		func = globals()[abstract_action.name]
39		observations, reward, done = func(self.agent_host)
40		time.sleep(0.2)
41		obs_list = []
42		if observations:
43		for k in observations:
44		if isinstance(observations[k], list):
45		obs_list.append(mk_evaluation(k, *observations[k]))
46		else:
47		obs_list.append(mk_evaluation(k, observations[k]))
48		except RuntimeError as e:
49		print("Error sending command:", e)
50		return obs_list, mk_evaluation("Reward", reward), done
51
52
53		class FlatworldAgent(OpencogAgent):
54		def __init__(self, env, atomspace):
55		set_default_atomspace(atomspace)
56
57		# Create Action Space. The set of allowed actions an agent can take.
58		# TODO take care of action parameters.
59		action_space = {
60		ExecutionLink(SchemaNode("go_to_key")),
61		ExecutionLink(SchemaNode("go_to_house")),
62		ExecutionLink(SchemaNode("go_to_diamonds")),
63		}
64		# Create Goal
65		pgoal = EvaluationLink(PredicateNode("Reward"), NumberNode("1"))
66		ngoal = EvaluationLink(PredicateNode("Reward"), NumberNode("0"))
67
68		# Call super ctor
69		OpencogAgent.__init__(self, env, atomspace, action_space, pgoal, ngoal)
70
71		# Overwrite some OpencogAgent parameters
72		self.polyaction_mining = False
73		self.monoaction_general_succeedent_mining = True
74		self.temporal_deduction = True
75		self.cogscm_minimum_strength = 0.75
76		self.cogscm_maximum_shannon_entropy = 1
77		self.cogscm_maximum_differential_entropy = 0
78		self.cogscm_maximum_variables = 0
79		self.miner_maximum_iterations = 100000
80		self.miner_maximum_variables = 9
81		self.miner_minimum_support = 3
82		self.conditional_conjunction_introduction = False
83		self.expiry = 3
84		self.visualize_cogscm = False
85		self.mixture_model.complexity_penalty = 0.6
86		self.mixture_model.delta = 1.0e-30
87		self.mixture_model.compressiveness = 0.1
88		self.mixture_model.weight_influence = 1.0
89
90		# Todo: restart the environment to get an initial reward
91		self.initial_reward = EvaluationLink(PredicateNode("Reward"), NumberNode("0"))
92
93
94		if __name__ == "__main__":
95		atomspace = AtomSpace()
96		set_default_atomspace(atomspace)
97
98		# Init loggers
99		log.set_level("info")
100		# log.set_sync(True)
101		agent_log.set_level("fine")
102		# agent_log.set_sync(True)
103		ure_logger().set_level("debug")
104		miner_log = MinerLogger(atomspace)
105		miner_log.set_level("debug")
106
107		#
108		wrapped_env = AbstractMalmoWrapper(missionXML=missionXML, validate=True)
109
110		cog_agent = FlatworldAgent(wrapped_env, atomspace)
111
112		cogscm_atomspace_file = "cogscms_as.scm"
113		saved_cogscm = os.path.isfile(cogscm_atomspace_file)
114		if cog_agent.visualize_cogscm:
115		# The address of the scheme REST endpoint
116		PORT, IP_ADDRESS = "5000", "127.0.0.1"
117		try:
118		status_code = requests.get(
119		"http://{}:{}/api/v1.1/atoms".format(IP_ADDRESS, PORT)
120		)
121		except:
122		raise RuntimeError(
123		"REST service is not started. Please run start_rest_service.py file first."
124		)
125
126		# Log all parameters of cag, useful for debugging
127		cog_agent.log_parameters(level="debug")
128		reward_t0 = cog_agent.record(cog_agent.initial_reward, 0, TRUE_TV)
129		agent_log.info("Initial reward: {}".format(reward_t0))
130
131		# Training/learning loop
132		lt_iterations = 2 # Number of learning-training iterations
133		lt_period = 50 # Duration of a learning-training iteration
134		for i in range(lt_iterations):
135		cog_agent.reset_action_counter()
136		par = cog_agent.accumulated_reward # Keep track of the reward before
137		if i > 0:
138		print(
139		"""
140		**************************************
141		* Applying PLN and Pattern Miner ... *
142		**************************************
143
144		"""
145		)
146		if saved_cogscm:
147		# load cogscm_atomspace
148		cog_agent.load_cogscms_atomspace(cogscm_atomspace_file)
149		else:
150		# Discover patterns to make more informed decisions
151		agent_log.info("Start learning ({}/{})".format(i + 1, lt_iterations))
152		cog_agent.learn()
153
154		# Visualize discovered cogscms
155		# set self.visualize_cogscm=True to enable visualization of the cogscms.
156		# Note that, RESTAPI needs be running to be able to post to the scheme endpoint.
157		# check start_rest_service.py file.
158		if cog_agent.visualize_cogscm:
159		post_to_restapi_scheme_endpoint(
160		None, cogscm_as=cog_agent.cogscms_atomspace
161		)
162
163		# Run agent to accumulate percepta
164		agent_log.info("Start training ({}/{})".format(i + 1, lt_iterations))
165		for j in range(lt_period):
166		cog_agent.control_cycle()
167		time.sleep(0.01)
168		log.info("step_count = {}".format(cog_agent.cycle_count))
169		nar = cog_agent.accumulated_reward - par
170		agent_log.info(
171		"Accumulated reward during {}th iteration = {}".format(i + 1, nar)
172		)
173		agent_log.info(
174		"Action counter during {}th iteration:\n{}".format(
175		i + 1, cog_agent.action_counter
176		)
177		)
178		# Log all agent's atomspaces at the end (at fine level)
179		agent_log_atomspace(cog_agent.atomspace, "fine", "cog_agent.atomspace")
180		agent_log_atomspace(
181		cog_agent.percepta_atomspace, "fine", "cog_agent.percepta_atomspace"
182		)
183		agent_log_atomspace(
184		cog_agent.cogscms_atomspace, "fine", "cog_agent.cogscms_atomspace"
185		)
186		agent_log_atomspace(
187		cog_agent.working_atomspace, "fine", "cog_agent.working_atomspace"
188		)
189
190		# Save cogscm_atomspace
191		if not saved_cogscm:
192		cog_agent.save_cogscms_atomspace("cogscms_as.scm")

examples/collect_diamonds/README.md +137 -0

…	…	@@ -0,0 +1,137 @@
	1	# Collect diamonds
	2
	3	## Description
	4
	5	Agent for a simple minecraft environment which includes blocks of
	6	diamonds locked inside a house and a key to the house. The agent must
	7	look for the key, open the house and collect a diamond to get a
	8	reward.
	9
	10	## Status
	11
	12	The Rocca agent is able to learn and plan the desired cognitive
	13	schematics leading to getting/accumulating a reward via pattern mining
	14	and temporal deduction.
	15
	16	The three optimal action plans required to get a reward are:
	17
	18	1. `outside(self, house) ∧ do(go_to_key) ↝ hold(self, key)`
	19
	20	```scheme
	21	(BackPredictiveImplicationScopeLink (stv 1 0.0111248)
	22	(VariableSet
	23	) ; [80075fffffff523a][1]
	24	(SLink
	25	(ZLink
	26	) ; [800fbffffffe8ce4][1]
	27	) ; [da5f815ba9d4009f][1]
	28	(AndLink
	29	(EvaluationLink (stv 0.64 0.0588235)
	30	(PredicateNode "outside") ; [72730412e28a734][1]
	31	(ListLink
	32	(ConceptNode "self") ; [40b11d11524bd751][1]
	33	(ConceptNode "house") ; [63eb9919f37daa5f][1]
	34	) ; [aadca36fe9d1a468][1]
	35	) ; [ca0c329fb1ab493b][1]
	36	(ExecutionLink
	37	(SchemaNode "go_to_key") ; [7f46c329a5e57604][1]
	38	) ; [f8086e6fdf73cdf4][1]
	39	) ; [ddda31153cf2aa6d][1]
	40	(EvaluationLink (stv 0.32 0.0588235)
	41	(PredicateNode "hold") ; [4b1b7a8b0a4d2853][1]
	42	(ListLink
	43	(ConceptNode "self") ; [40b11d11524bd751][1]
	44	(ConceptNode "key") ; [4d0844146f96d3][1]
	45	) ; [e7a9c95ae7484b28][1]
	46	) ; [e6a8f21e6b37d8f0][1]
	47	) ; [ff069058911f0233][1]
	48	```
	49	2. `outside(self, house) ∧ hold(self, key) ∧ do(go_to_house) ↝ inside(self, house)`
	50
	51	```scheme
	52	(BackPredictiveImplicationScopeLink (stv 1 0.00621118)
	53	(VariableSet
	54	) ; [80075fffffff523a][1]
	55	(SLink
	56	(ZLink
	57	) ; [800fbffffffe8ce4][1]
	58	) ; [da5f815ba9d4009f][1]
	59	(AndLink
	60	(ExecutionLink
	61	(SchemaNode "go_to_house") ; [7e1737e3e117d059][1]
	62	) ; [93427319ec122fff][1]
	63	(EvaluationLink (stv 0.64 0.0588235)
	64	(PredicateNode "outside") ; [72730412e28a734][1]
	65	(ListLink
	66	(ConceptNode "self") ; [40b11d11524bd751][1]
	67	(ConceptNode "house") ; [63eb9919f37daa5f][1]
	68	) ; [aadca36fe9d1a468][1]
	69	) ; [ca0c329fb1ab493b][1]
	70	(EvaluationLink (stv 0.32 0.0588235)
	71	(PredicateNode "hold") ; [4b1b7a8b0a4d2853][1]
	72	(ListLink
	73	(ConceptNode "self") ; [40b11d11524bd751][1]
	74	(ConceptNode "key") ; [4d0844146f96d3][1]
	75	) ; [e7a9c95ae7484b28][1]
	76	) ; [e6a8f21e6b37d8f0][1]
	77	) ; [fbf4892ec0643e2c][1]
	78	(EvaluationLink
	79	(PredicateNode "inside") ; [63398dcfcf85c8a3][1]
	80	(ListLink
	81	(ConceptNode "self") ; [40b11d11524bd751][1]
	82	(ConceptNode "house") ; [63eb9919f37daa5f][1]
	83	) ; [aadca36fe9d1a468][1]
	84	) ; [871c182b52e89756][1]
	85	) ; [a2984b519c1ddc4f][1]
	86	```
	87
	88	3. `inside(self, house) ∧ do(go_to_diamonds) ↝ Reward(1)`
	89
	90	```scheme
	91	(BackPredictiveImplicationScopeLink (stv 1 0.00621118)
	92	(VariableSet
	93	) ; [80075fffffff523a][1]
	94	(SLink
	95	(ZLink
	96	) ; [800fbffffffe8ce4][1]
	97	) ; [da5f815ba9d4009f][1]
	98	(AndLink
	99	(EvaluationLink
	100	(PredicateNode "inside") ; [63398dcfcf85c8a3][1]
	101	(ListLink
	102	(ConceptNode "self") ; [40b11d11524bd751][1]
	103	(ConceptNode "house") ; [63eb9919f37daa5f][1]
	104	) ; [aadca36fe9d1a468][1]
	105	) ; [871c182b52e89756][1]
	106	(ExecutionLink
	107	(SchemaNode "go_to_diamonds") ; [7aee74cf6bad6442][1]
	108	) ; [a2630f96ffbe0861][1]
	109	) ; [b4373ec3773f1783][1]
	110	(EvaluationLink
	111	(PredicateNode "Reward") ; [155bb4d713db0d51][1]
	112	(NumberNode "1") ; [2cf0956d543cff8e][1]
	113	) ; [d3cee8bdda06ffcb][1]
	114	) ; [8a298365b46b204b][1]
	115	```
	116
	117	## Usage
	118
	119	1. Clone malmo https://github.com/Microsoft/malmo
	120
	121	Note: No need to compile malmo as a pre-built version (MalmoPython)
	122	has been included under `../rocca/malmo`. Only install the required
	123	dependencies to launch minecraft.
	124
	125	2. Launch `<MALMO>/Minecraft/launchClient.sh`.
	126
	127	3. Launch collect diamonds `python collect_diamonds.py`.
	128
	129	- Note: If there is a need to visualize mined cognitive schematics,
	130	do the following before step 3.
	131
	132	- Build and run atompace-explorer here
	133	https://github.com/tanksha/atomspace-explorer rocca_demo branch.
	134
	135	- Start the RESTAPI `python start_rest_service.py`
	136
	137	- Open `collect_diamonds.py` and set `self.visualize_cogscm = True`

examples/collect_diamonds/__init__.py +0 -0

(no content changes — binary or rename)

examples/collect_diamonds/collect_diamonds.py +192 -0

…	…	@@ -0,0 +1,192 @@
	1	import time
	2
	3	# OpenCog
	4	from opencog.logger import log
	5	from opencog.pln import *
	6	from opencog.type_constructors import *
	7	from opencog.utilities import set_default_atomspace
	8	from opencog.ure import ure_logger
	9
	10
	11	from rocca.agents import OpencogAgent
	12	from rocca.agents.utils import TRUE_TV, agent_log
	13	from rocca.envs.malmo_demo.collect_diamonds_env import *
	14	from rocca.envs.wrappers import MalmoWrapper
	15	from rocca.envs.wrappers.utils import mk_evaluation
	16	from rocca.agents.utils import *
	17	from functools import wraps
	18	import os
	19	import requests
	20
	21
	22	class AbstractMalmoWrapper(MalmoWrapper):
	23	def __init__(self, missionXML, validate):
	24	MalmoWrapper.__init__(self, missionXML, validate)
	25
	26	@staticmethod
	27	def step_decorator(step):
	28	@wraps(step)
	29	def wrapper(ref, action, update_callback=None):
	30	name = action.out[0]
	31	return step(ref, name, update_callback)
	32
	33	return wrapper
	34
	35	@step_decorator.__func__
	36	def step(self, abstract_action, update_callback=None):
	37	try:
	38	func = globals()[abstract_action.name]
	39	observations, reward, done = func(self.agent_host)
	40	time.sleep(0.2)
	41	obs_list = []
	42	if observations:
	43	for k in observations:
	44	if isinstance(observations[k], list):
	45	obs_list.append(mk_evaluation(k, *observations[k]))
	46	else:
	47	obs_list.append(mk_evaluation(k, observations[k]))
	48	except RuntimeError as e:
	49	print("Error sending command:", e)
	50	return obs_list, mk_evaluation("Reward", reward), done
	51
	52
	53	class FlatworldAgent(OpencogAgent):
	54	def __init__(self, env, atomspace):
	55	set_default_atomspace(atomspace)
	56
	57	# Create Action Space. The set of allowed actions an agent can take.
	58	# TODO take care of action parameters.
	59	action_space = {
	60	ExecutionLink(SchemaNode("go_to_key")),
	61	ExecutionLink(SchemaNode("go_to_house")),
	62	ExecutionLink(SchemaNode("go_to_diamonds")),
	63	}
	64	# Create Goal
	65	pgoal = EvaluationLink(PredicateNode("Reward"), NumberNode("1"))
	66	ngoal = EvaluationLink(PredicateNode("Reward"), NumberNode("0"))
	67
	68	# Call super ctor
	69	OpencogAgent.__init__(self, env, atomspace, action_space, pgoal, ngoal)
	70
	71	# Overwrite some OpencogAgent parameters
	72	self.polyaction_mining = False
	73	self.monoaction_general_succeedent_mining = True
	74	self.temporal_deduction = True
	75	self.cogscm_minimum_strength = 0.75
	76	self.cogscm_maximum_shannon_entropy = 1
	77	self.cogscm_maximum_differential_entropy = 0
	78	self.cogscm_maximum_variables = 0
	79	self.miner_maximum_iterations = 100000
	80	self.miner_maximum_variables = 9
	81	self.miner_minimum_support = 3
	82	self.conditional_conjunction_introduction = False
	83	self.expiry = 3
	84	self.visualize_cogscm = False
	85	self.mixture_model.complexity_penalty = 0.6
	86	self.mixture_model.delta = 1.0e-30
	87	self.mixture_model.compressiveness = 0.1
	88	self.mixture_model.weight_influence = 1.0
	89
	90	# Todo: restart the environment to get an initial reward
	91	self.initial_reward = EvaluationLink(PredicateNode("Reward"), NumberNode("0"))
	92
	93
	94	if __name__ == "__main__":
	95	atomspace = AtomSpace()
	96	set_default_atomspace(atomspace)
	97
	98	# Init loggers
	99	log.set_level("info")
	100	# log.set_sync(True)
	101	agent_log.set_level("fine")
	102	# agent_log.set_sync(True)
	103	ure_logger().set_level("debug")
	104	miner_log = MinerLogger(atomspace)
	105	miner_log.set_level("debug")
	106
	107	#
	108	wrapped_env = AbstractMalmoWrapper(missionXML=missionXML, validate=True)
	109
	110	cog_agent = FlatworldAgent(wrapped_env, atomspace)
	111
	112	cogscm_atomspace_file = "cogscms_as.scm"
	113	saved_cogscm = os.path.isfile(cogscm_atomspace_file)
	114	if cog_agent.visualize_cogscm:
	115	# The address of the scheme REST endpoint
	116	PORT, IP_ADDRESS = "5000", "127.0.0.1"
	117	try:
	118	status_code = requests.get(
	119	"http://{}:{}/api/v1.1/atoms".format(IP_ADDRESS, PORT)
	120	)
	121	except:
	122	raise RuntimeError(
	123	"REST service is not started. Please run start_rest_service.py file first."
	124	)
	125
	126	# Log all parameters of cag, useful for debugging
	127	cog_agent.log_parameters(level="debug")
	128	reward_t0 = cog_agent.record(cog_agent.initial_reward, 0, TRUE_TV)
	129	agent_log.info("Initial reward: {}".format(reward_t0))
	130
	131	# Training/learning loop
	132	lt_iterations = 2 # Number of learning-training iterations
	133	lt_period = 50 # Duration of a learning-training iteration
	134	for i in range(lt_iterations):
	135	cog_agent.reset_action_counter()
	136	par = cog_agent.accumulated_reward # Keep track of the reward before
	137	if i > 0:
	138	print(
	139	"""
	140	**************************************
	141	* Applying PLN and Pattern Miner ... *
	142	**************************************
	143
	144	"""
	145	)
	146	if saved_cogscm:
	147	# load cogscm_atomspace
	148	cog_agent.load_cogscms_atomspace(cogscm_atomspace_file)
	149	else:
	150	# Discover patterns to make more informed decisions
	151	agent_log.info("Start learning ({}/{})".format(i + 1, lt_iterations))
	152	cog_agent.learn()
	153
	154	# Visualize discovered cogscms
	155	# set self.visualize_cogscm=True to enable visualization of the cogscms.
	156	# Note that, RESTAPI needs be running to be able to post to the scheme endpoint.
	157	# check start_rest_service.py file.
	158	if cog_agent.visualize_cogscm:
	159	post_to_restapi_scheme_endpoint(
	160	None, cogscm_as=cog_agent.cogscms_atomspace
	161	)
	162
	163	# Run agent to accumulate percepta
	164	agent_log.info("Start training ({}/{})".format(i + 1, lt_iterations))
	165	for j in range(lt_period):
	166	cog_agent.control_cycle()
	167	time.sleep(0.01)
	168	log.info("step_count = {}".format(cog_agent.cycle_count))
	169	nar = cog_agent.accumulated_reward - par
	170	agent_log.info(
	171	"Accumulated reward during {}th iteration = {}".format(i + 1, nar)
	172	)
	173	agent_log.info(
	174	"Action counter during {}th iteration:\n{}".format(
	175	i + 1, cog_agent.action_counter
	176	)
	177	)
	178	# Log all agent's atomspaces at the end (at fine level)
	179	agent_log_atomspace(cog_agent.atomspace, "fine", "cog_agent.atomspace")
	180	agent_log_atomspace(
	181	cog_agent.percepta_atomspace, "fine", "cog_agent.percepta_atomspace"
	182	)
	183	agent_log_atomspace(
	184	cog_agent.cogscms_atomspace, "fine", "cog_agent.cogscms_atomspace"
	185	)
	186	agent_log_atomspace(
	187	cog_agent.working_atomspace, "fine", "cog_agent.working_atomspace"
	188	)
	189
	190	# Save cogscm_atomspace
	191	if not saved_cogscm:
	192	cog_agent.save_cogscms_atomspace("cogscms_as.scm")

examples/collect_diamonds/start_rest_service.py +26 -0

…	…	@@ -0,0 +1,26 @@
	1	from opencog.web.api.apimain import RESTAPI
	2	from opencog.atomspace import AtomSpace
	3	from opencog.type_constructors import set_default_atomspace
	4	from opencog.scheme import scheme_eval
	5
	6	from requests import post
	7
	8
	9	def start_restapi(PORT="5000", IP_ADDRESS="127.0.0.1"):
	10	atomspace = AtomSpace()
	11	set_default_atomspace(atomspace)
	12	load_modules(atomspace)
	13	api = RESTAPI(atomspace)
	14	api.run(host=IP_ADDRESS, port=PORT)
	15
	16
	17	def load_modules(atomspace):
	18	scheme_eval(atomspace, "(use-modules (opencog))")
	19	scheme_eval(atomspace, "(use-modules (opencog exec))")
	20	scheme_eval(atomspace, "(use-modules (opencog pln))")
	21	scheme_eval(atomspace, "(use-modules (opencog spacetime))")
	22	scheme_eval(atomspace, "(use-modules (opencog persist))")
	23
	24
	25	if __name__ == "__main__":
	26	start_restapi()

examples/pong.py +0 -173

…	…	@@ -1,173 +0,0 @@
1		# Atari Pong Gym env
2		#
3		# WARNING: unmaintained
4
5		import gym
6		import random
7		import matplotlib.pyplot as plt
8		import numpy as np
9		from opencog.type_constructors import *
10		from opencog.spacetime import *
11
12		atomspace = AtomSpace()
13		set_default_atomspace(atomspace)
14
15		atomese_action_space = (
16		SchemaNode("Noop"),
17		SchemaNode("Fire"),
18		SchemaNode("Go Right"),
19		SchemaNode("Go Left"),
20		SchemaNode("Fire Right"),
21		SchemaNode("Fire Left"),
22		)
23
24		ATOMESE_TO_KEYS = {
25		"Noop": 0,
26		"Fire": 1,
27		"Go Right": 2,
28		"Go Left": 3,
29		"Fire Right": 4,
30		"Fire Left": 5,
31		}
32
33
34		def atomese_action_sample():
35		return atomese_action_space[random.randint(0, 5)]
36
37
38		def to_atlocation(cord, obj):
39		"""
40		Accepts center of Player, ball and opponent
41		location convert and pass it to AtLocationLink
42		Each object will be stored as such:
43
44		(AtLocationLink
45		(ConceptNode "Ball")
46		(ListLink
47		(ConceptNode "Pong"))
48		(ListLink
49		(NumberNode 45)
50		(NumberNode 76)
51		(NumberNode 0)))
52
53		returns a list of AtlocationLink for one frame
54		"""
55		tmp = [NumberNode(str(n)) for n in cord]
56		schema = AtLocationLink(
57		ConceptNode(str(obj)),
58		ListLink(ConceptNode("Pong"), ListLink(tmp[0], tmp[1], NumberNode("0"))),
59		)
60
61		return schema
62
63
64		def preprocess(obs):
65		"""
66		Converts RGB observation into black and white
67		white pixels represents the positon of ball and paddles
68
69		"""
70		obs = obs[35:195][0:-1]
71		obs[obs == 144] = 0
72		obs[obs == 72] = 0
73		obs[obs == 17] = 0
74		return obs
75
76
77		def observation_to_atomese(observation):
78		"""
79		Accepts observation from gym and converts
80		it to atomese. Note that we convert only the
81		white pixels.i.e the the x and y corrdinates
82		of all white pixels in the game which represents
83		the position of the paddle and the ball
84
85		returns a List of EvaluationLinks
86		"""
87		obs = preprocess(observation)
88		player = []
89		ball = []
90		opp = []
91		Player = np.array([92, 186, 92])
92		Ball = np.array([236, 236, 236])
93		Opponent = np.array([213, 130, 74])
94
95		for i in range(159):
96		for j in range(160):
97		temp = obs[i][j]
98		if (temp == Player).all():
99		player.append([j, i])
100		elif (temp == Ball).all():
101		ball.append([j, i])
102
103		elif (temp == Opponent).all():
104		opp.append([j, i])
105		else:
106		pass
107
108		beg = player[0]
109		end = player.pop()
110		midplayer = ((beg[0] + end[0]) / 2, (beg[1] + end[1]) / 2)
111		schemaplayer = to_atlocation(midplayer, "Player")
112		schemaball = SchemaNode("Null")
113		schemaopp = SchemaNode("Null")
114
115		if len(ball) != 0:
116		beg = ball[0]
117		end = ball.pop()
118		midball = ((beg[0] + end[0]) / 2, (beg[1] + end[1]) / 2)
119		schemaball = to_atlocation(midball, "Ball")
120
121		if len(opp) != 0:
122		beg = opp[0]
123		end = opp.pop()
124		midopp = ((beg[0] + end[0]) / 2, (beg[1] + end[1]) / 2)
125		schemaopp = to_atlocation(midopp, "Opponent")
126		return [schemaplayer, schemaball, schemaopp]
127
128
129		def reward_to_atomese(reward):
130		rn = NumberNode(str(reward))
131		return EvaluationLink(PredicateNode("Reward"), rn)
132
133
134		def action_to_gym(action):
135		"""
136		Converts atomese type action into gym
137		"""
138		if SchemaNode("Noop") == action:
139		return 0
140		if SchemaNode("Fire") == action:
141		return 1
142		if SchemaNode("Go Right") == action:
143		return 2
144		if SchemaNode("Go Left") == action:
145		return 3
146		if SchemaNode("Fire Right") == action:
147		return 4
148		if SchemaNode("Fire Left") == action:
149		return 5
150
151
152		def dummy_atomese_agent(atomese_obser):
153		"""
154		did not use observation at all
155		Dummy agent
156		"""
157		rand = random.randint(0, 5)
158		return atomese_action_space[rand]
159
160
161		if __name__ == "__main__":
162		env = gym.make("Pong-v0")
163		for i in range(100):
164		env.reset()
165		action = atomese_action_sample()
166		for j in range(100):
167		env.render()
168		gym_action = action_to_gym(action)
169		observation, reward, done, info = env.step(gym_action)
170		atomese_ops = observation_to_atomese(observation)
171		action = dummy_atomese_agent(atomese_ops)
172
173		env.close()

examples/pong/README.md +46 -0

…	…	@@ -0,0 +1,46 @@
	1	# Pong
	2
	3	## Description
	4
	5	Agent for Pong-v0 gym env.
	6
	7	## Status
	8
	9	Unmantained. Would be nice to maintain it though, it is a simple yet
	10	already challenging environment due to the delay between actions and
	11	rewards. I.e. the agent needs to move the paddle early on in order to
	12	avoid missing the ball later on.
	13
	14	## Requirements
	15
	16	You need to install the `atari` subpackage of gym, which you can do with
	17
	18	```
	19	pip install gym[atari]
	20	```
	21
	22	Additionally you need to install the Atari ROMS, download a ROM pack
	23	from
	24
	25	http://www.atarimania.com/rom_collection_archive_atari_2600_roms.html
	26
	27	unrar it under the current folder
	28
	29	```
	30	unrar e Roms.rar
	31	```
	32
	33	It should create 2 zip files. Then you can install the ROMS with the
	34	following command
	35
	36	```
	37	python -m atari_py.import_roms <ROM_FOLDER>
	38	```
	39
	40	You'll need to have `atari-py` properly installed for that.
	41
	42	## Usage
	43
	44	```bash
	45	python pong.py
	46	```

examples/pong/pong.py +173 -0

…	…	@@ -0,0 +1,173 @@
	1	# Atari Pong Gym env
	2	#
	3	# WARNING: unmaintained
	4
	5	import gym
	6	import random
	7	import matplotlib.pyplot as plt
	8	import numpy as np
	9	from opencog.type_constructors import *
	10	from opencog.spacetime import *
	11
	12	atomspace = AtomSpace()
	13	set_default_atomspace(atomspace)
	14
	15	atomese_action_space = (
	16	SchemaNode("Noop"),
	17	SchemaNode("Fire"),
	18	SchemaNode("Go Right"),
	19	SchemaNode("Go Left"),
	20	SchemaNode("Fire Right"),
	21	SchemaNode("Fire Left"),
	22	)
	23
	24	ATOMESE_TO_KEYS = {
	25	"Noop": 0,
	26	"Fire": 1,
	27	"Go Right": 2,
	28	"Go Left": 3,
	29	"Fire Right": 4,
	30	"Fire Left": 5,
	31	}
	32
	33
	34	def atomese_action_sample():
	35	return atomese_action_space[random.randint(0, 5)]
	36
	37
	38	def to_atlocation(cord, obj):
	39	"""
	40	Accepts center of Player, ball and opponent
	41	location convert and pass it to AtLocationLink
	42	Each object will be stored as such:
	43
	44	(AtLocationLink
	45	(ConceptNode "Ball")
	46	(ListLink
	47	(ConceptNode "Pong"))
	48	(ListLink
	49	(NumberNode 45)
	50	(NumberNode 76)
	51	(NumberNode 0)))
	52
	53	returns a list of AtlocationLink for one frame
	54	"""
	55	tmp = [NumberNode(str(n)) for n in cord]
	56	schema = AtLocationLink(
	57	ConceptNode(str(obj)),
	58	ListLink(ConceptNode("Pong"), ListLink(tmp[0], tmp[1], NumberNode("0"))),
	59	)
	60
	61	return schema
	62
	63
	64	def preprocess(obs):
	65	"""
	66	Converts RGB observation into black and white
	67	white pixels represents the positon of ball and paddles
	68
	69	"""
	70	obs = obs[35:195][0:-1]
	71	obs[obs == 144] = 0
	72	obs[obs == 72] = 0
	73	obs[obs == 17] = 0
	74	return obs
	75
	76
	77	def observation_to_atomese(observation):
	78	"""
	79	Accepts observation from gym and converts
	80	it to atomese. Note that we convert only the
	81	white pixels.i.e the the x and y corrdinates
	82	of all white pixels in the game which represents
	83	the position of the paddle and the ball
	84
	85	returns a List of EvaluationLinks
	86	"""
	87	obs = preprocess(observation)
	88	player = []
	89	ball = []
	90	opp = []
	91	Player = np.array([92, 186, 92])
	92	Ball = np.array([236, 236, 236])
	93	Opponent = np.array([213, 130, 74])
	94
	95	for i in range(159):
	96	for j in range(160):
	97	temp = obs[i][j]
	98	if (temp == Player).all():
	99	player.append([j, i])
	100	elif (temp == Ball).all():
	101	ball.append([j, i])
	102
	103	elif (temp == Opponent).all():
	104	opp.append([j, i])
	105	else:
	106	pass
	107
	108	beg = player[0]
	109	end = player.pop()
	110	midplayer = ((beg[0] + end[0]) / 2, (beg[1] + end[1]) / 2)
	111	schemaplayer = to_atlocation(midplayer, "Player")
	112	schemaball = SchemaNode("Null")
	113	schemaopp = SchemaNode("Null")
	114
	115	if len(ball) != 0:
	116	beg = ball[0]
	117	end = ball.pop()
	118	midball = ((beg[0] + end[0]) / 2, (beg[1] + end[1]) / 2)
	119	schemaball = to_atlocation(midball, "Ball")
	120
	121	if len(opp) != 0:
	122	beg = opp[0]
	123	end = opp.pop()
	124	midopp = ((beg[0] + end[0]) / 2, (beg[1] + end[1]) / 2)
	125	schemaopp = to_atlocation(midopp, "Opponent")
	126	return [schemaplayer, schemaball, schemaopp]
	127
	128
	129	def reward_to_atomese(reward):
	130	rn = NumberNode(str(reward))
	131	return EvaluationLink(PredicateNode("Reward"), rn)
	132
	133
	134	def action_to_gym(action):
	135	"""
	136	Converts atomese type action into gym
	137	"""
	138	if SchemaNode("Noop") == action:
	139	return 0
	140	if SchemaNode("Fire") == action:
	141	return 1
	142	if SchemaNode("Go Right") == action:
	143	return 2
	144	if SchemaNode("Go Left") == action:
	145	return 3
	146	if SchemaNode("Fire Right") == action:
	147	return 4
	148	if SchemaNode("Fire Left") == action:
	149	return 5
	150
	151
	152	def dummy_atomese_agent(atomese_obser):
	153	"""
	154	did not use observation at all
	155	Dummy agent
	156	"""
	157	rand = random.randint(0, 5)
	158	return atomese_action_space[rand]
	159
	160
	161	if __name__ == "__main__":
	162	env = gym.make("Pong-v0")
	163	for i in range(100):
	164	env.reset()
	165	action = atomese_action_sample()
	166	for j in range(100):
	167	env.render()
	168	gym_action = action_to_gym(action)
	169	observation, reward, done, info = env.step(gym_action)
	170	atomese_ops = observation_to_atomese(observation)
	171	action = dummy_atomese_agent(atomese_ops)
	172
	173	env.close()

examples/start_rest_service.py +0 -26

…	…	@@ -1,26 +0,0 @@
1		from opencog.web.api.apimain import RESTAPI
2		from opencog.atomspace import AtomSpace
3		from opencog.type_constructors import set_default_atomspace
4		from opencog.scheme import scheme_eval
5
6		from requests import post
7
8
9		def start_restapi(PORT="5000", IP_ADDRESS="127.0.0.1"):
10		atomspace = AtomSpace()
11		set_default_atomspace(atomspace)
12		load_modules(atomspace)
13		api = RESTAPI(atomspace)
14		api.run(host=IP_ADDRESS, port=PORT)
15
16
17		def load_modules(atomspace):
18		scheme_eval(atomspace, "(use-modules (opencog))")
19		scheme_eval(atomspace, "(use-modules (opencog exec))")
20		scheme_eval(atomspace, "(use-modules (opencog pln))")
21		scheme_eval(atomspace, "(use-modules (opencog spacetime))")
22		scheme_eval(atomspace, "(use-modules (opencog persist))")
23
24
25		if __name__ == "__main__":
26		start_restapi()

opencog-rocca commits / c9c0833

Changed files (20) +1112 -1081