Showing
5 changed files
with
359 additions
and
0 deletions
object_detection/imgs/1_1.PNG
0 → 100644
336 KB
object_detection/imgs/1_2.jpg
0 → 100644
14.5 KB
object_detection/imgs/1_3.PNG
0 → 100644
1.17 MB
object_detection/imgs/1_4.PNG
0 → 100644
139 KB
object_detection/make_object_detector.md
0 → 100644
1 | +# How to make your own object detector | ||
2 | + | ||
3 | +> ### with TF (TensorFlow) Object Detection API & GCP (Google Gloud Platform) | ||
4 | + | ||
5 | +EastFull: LEE DONGCHAN (huromech@gmail.com) | ||
6 | + | ||
7 | + | ||
8 | + | ||
9 | +## TF Object Detection API and GCP | ||
10 | + | ||
11 | +![1_1](./imgs/1_1.PNG) | ||
12 | + | ||
13 | + | ||
14 | + | ||
15 | + Object detection is the task of recognizing what class an objects is and where it is located. So, this may be more complex and difficult to implement than a simple classification model. [TensorFlow Object Detection API](https://github.com/tensorflow/models/tree/master/research/object_detection) uses pre-trained object detection model to make it easier to implement object detection functions. In addition, the model can be trained in the cloud using [Google Cloud Platform](https://cloud.google.com/gcp/?utm_source=google&utm_medium=cpc&utm_campaign=na-US-all-en-dr-skws-all-all-trial-b-dr-1008076&utm_content=text-ad-none-any-DEV_c-CRE_109860918967-ADGP_Hybrid+%7C+AW+SEM+%7C+SKWS+%7C+US+%7C+en+%7C+Multi+~+Cloud-KWID_43700009609890930-kwd-19383198255&utm_term=KW_%2Bcloud-ST_%2Bcloud&gclid=CjwKCAiA4Y7yBRB8EiwADV1haezYvLCzbz2c6lJvwoLzlRAcCk7YaVJemA3VasM2MR4axqgmG02MqRoCS48QAvD_BwE) without configuring the hardware environment for learning. Therefore, this article will talk about how to make your own object detector using TensorFlow Object Detection API and Google Cloud Platform :D | ||
16 | + | ||
17 | + | ||
18 | + | ||
19 | +## Tabel of Contents | ||
20 | + | ||
21 | +0. Setting up the Google Cloud Platform | ||
22 | +1. Installing TensorFlow and TensorFlow Object Detection API | ||
23 | +2. Making dataset | ||
24 | +3. Using the pre-trained SSD for Transfer Learning | ||
25 | +4. Training and Evaluation Jobs on Google Cloud ML Engine | ||
26 | + | ||
27 | + | ||
28 | + | ||
29 | +## 0. Setting up the Google Cloud Platform | ||
30 | + | ||
31 | + Before we begin, we will do the following to use Google Cloud ML engine and Storage. | ||
32 | + | ||
33 | +> 1. [Create a project](https://cloud.google.com/resource-manager/docs/creating-managing-projects) | ||
34 | +> 2. [Modify a project's billing settings](https://cloud.google.com/billing/docs/how-to/modify-project) | ||
35 | +> 3. [Install the Google Cloud SDK](https://cloud.google.com/sdk/install) to use Google Cloud Storage and ML Engine in your workplace (*In my case, I installed Ubuntu 18.04 LTS terminal in Windows 10. For the installaion of terminal, see [Terminal tutorial at fast.ai](https://course.fast.ai/terminal_tutorial.html) and the installation of Goolge Cloud CLI, see [here](https://course.fast.ai/start_gcp.html#step-2-install-google-cli) at course of fast.ai*) | ||
36 | +> 4. [Enable the ML Engine APIs](https://console.cloud.google.com/flows/enableapi?apiid=ml.googleapis.com,compute_component&_ga=2.43515109.-1978295503.1509743045) | ||
37 | +> 5. [Set up a Google Cloud Storage bucket](https://cloud.google.com/storage/docs/creating-buckets) (*Or, you can use the following command in the Google Cloud CLI*:`gsutil mb gs://YOUR_UNIQUE_BUCKET_NAME`) | ||
38 | + | ||
39 | + | ||
40 | + | ||
41 | + Next, you need to add a TPU-specific service account to give the cloud TPU access to the project. Enter the following command to get the name of the account: | ||
42 | + | ||
43 | +``` | ||
44 | +curl -H "Authorization: Bearer $(gcloud auth print-access-token) \ | ||
45 | + https://ml.googleapis.com/v1/projects/YOUR_PROJECT_ID:getConfig | ||
46 | +``` | ||
47 | + | ||
48 | + | ||
49 | + | ||
50 | +After command, copy the `tpuServiceAccount`(it looks like `service-account-12345@cloud-tpu.iam.gserviceaccount.com`) and save then as environment variable: | ||
51 | + | ||
52 | +``` | ||
53 | +export TPU_ACCOUNT=your-service-account | ||
54 | +``` | ||
55 | + | ||
56 | + | ||
57 | + | ||
58 | +And, give the role of ML serviceAgent to the TPU service account: | ||
59 | + | ||
60 | +``` | ||
61 | +gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \ | ||
62 | + --member serviceAccount:$TPU_ACCOUNT --role roles/ml.serviceAgent | ||
63 | +``` | ||
64 | + | ||
65 | + | ||
66 | + | ||
67 | +## 1. Installing TensorFlow and TensorFlow Object Detection API | ||
68 | + | ||
69 | + You can install the TensorFlow Object Detection API by following [this installaion](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/installation.md). Please refer to [here](https://www.tensorflow.org/install/) for more detailed procedures on TensorFlow. (*Notes: In my case, the version under tensorflow 1.13 and tensorflow 2.x did not work properly. I used the tensorflow 1.14.0 in a number of attempts :D - as of Feb, 2020*) | ||
70 | + | ||
71 | + | ||
72 | + | ||
73 | +## 2. Making dataset | ||
74 | + | ||
75 | +### 2.1. gathering data | ||
76 | + | ||
77 | +![1_2](./imgs/1_2.jpg) | ||
78 | + | ||
79 | + There is still one more important process left! We need data before we build our detection model. In my case, I have collected several data to find a badprint that frequently occurs in 3D printing. Collect data for your object detector :D | ||
80 | + | ||
81 | + | ||
82 | + | ||
83 | + ### 2.2. labeling data | ||
84 | + | ||
85 | +![1_3](./imgs/1_3.PNG) | ||
86 | + | ||
87 | + Now, we label the data that we have collected. I used [VoTT (Visual Object Tagging Tool)](https://github.com/microsoft/VoTT) to carry out labelling and saved it in the form of a `.csv` file. | ||
88 | + | ||
89 | +In fact, it doesn't matter which labelling tool you use. However, if yo want to follow this article to configure the dataset, the information (about `filename`, `xmin`, `ymin`, `xmax`, `ymax`, `class`) must be specified in the `.csv` file. | ||
90 | + | ||
91 | + | ||
92 | + | ||
93 | +### 2.3. generating TFRecords | ||
94 | + | ||
95 | + In order to train the object detector, we have to create [TFRecords](https://www.tensorflow.org/tutorials/load_data/tfrecord). To do this, we will use `generate_tfrecord.py` from [datitran's raccon_dataset](https://github.com/datitran/raccoon_dataset). Before make TFRecords we need to change a few lines in the `generate_tfrecord.py`. | ||
96 | + | ||
97 | +From: | ||
98 | + | ||
99 | +```python | ||
100 | +# TO-DO replace this with label map | ||
101 | +def class_text_to_int(row_label): | ||
102 | + if row_label == 'raccoon': | ||
103 | + return 1 | ||
104 | + else: | ||
105 | + None | ||
106 | +``` | ||
107 | + | ||
108 | + | ||
109 | + | ||
110 | +To: | ||
111 | + | ||
112 | +```python | ||
113 | +# TO-DO replace this with label map | ||
114 | +def class_text_to_int(row_label): | ||
115 | + if row_label == 'bad': | ||
116 | + return 1 | ||
117 | + elif row_label == 'WHAT YOU WANT !': | ||
118 | + return 2 | ||
119 | + #. | ||
120 | + #. | ||
121 | + #. | ||
122 | + else: | ||
123 | + None | ||
124 | +``` | ||
125 | + | ||
126 | +If you are using a different dataset, you must replace the class name. | ||
127 | + | ||
128 | + | ||
129 | + | ||
130 | +Now our TFRecords can be generated by: | ||
131 | + | ||
132 | +``` | ||
133 | +python generate_tfrecord.py \ | ||
134 | + --csv_input=YOUR_PATH/TRAIN_CSV_NAME.csv \ | ||
135 | + --output_path=YOUR_PATH/train.record \ | ||
136 | + --image_dir=YOUR_PATH/IMAGE_DIR | ||
137 | + | ||
138 | +python generate_tfrecord.py \ | ||
139 | + --csv_input=YOUR_PATH/VALID_CSV_NAME.csv \ | ||
140 | + --output_path=YOUR_PATH/val.record \ | ||
141 | + --image_dir=YOUR_PATH/IMAGE_DIR | ||
142 | +``` | ||
143 | + | ||
144 | + | ||
145 | + | ||
146 | +Next, we will place the `.tfrecord` on our Google Cloud Storage bucket. | ||
147 | + | ||
148 | +``` | ||
149 | +gsutil -m cp -r YOUR_PATH/train.record gs://YOUR_UNIQUE_BUCKET_NAME/data/ | ||
150 | +gsutil -m cp -r YOUR_PATH/val.record gs://YOUR_UNIQUE_BUCKET_NAME/data/ | ||
151 | +``` | ||
152 | + | ||
153 | + | ||
154 | + | ||
155 | +### 2.4. creating a label map | ||
156 | + | ||
157 | + The label map maps an id to a name. This file is in `.pbtxt` format. In my case, I made it using the vim editor in terminal. | ||
158 | + | ||
159 | +``` | ||
160 | +item { | ||
161 | + id: 1 | ||
162 | + name: 'bad' | ||
163 | +} | ||
164 | + | ||
165 | +item { | ||
166 | + id: 2 | ||
167 | + name: 'WHAT YOU WANT !' | ||
168 | +} | ||
169 | + | ||
170 | + . | ||
171 | + . | ||
172 | + . | ||
173 | +``` | ||
174 | + | ||
175 | + | ||
176 | + | ||
177 | +After creating a label map, we will place the `.pbtxt` on our Google Cloud Storage bucket. | ||
178 | + | ||
179 | +``` | ||
180 | +gsutil -m cp -r YOUR_PATH/LABEL_MAP.pbtxt gs://YOUR_UNIQUE_BUCKET_NAME/data/ | ||
181 | +``` | ||
182 | + | ||
183 | + | ||
184 | + | ||
185 | +## 3. Using the pre-trained SSD for Transfer Learning | ||
186 | + | ||
187 | +### 3.1. downloading model | ||
188 | + | ||
189 | + There are many pre-trained models we can use. We will use the checkpoints from these pre-trained models and apply to our task. What makes this possible is Transfer Learning. This is a strategy that enables pretty good performance with less time and data. Anyway, please check out the [Object Detection Models Zoo](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md). | ||
190 | + | ||
191 | +``` | ||
192 | +wget http://storage.googleapis.com/download.tensorflow.org/models/object_detection/ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03.tar.gz | ||
193 | + | ||
194 | +tar xzf ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03.tar.gz | ||
195 | +``` | ||
196 | + | ||
197 | +So here we're going to use the ssd model based on the mobilenet. If you want a different model, try it! After downloading the model, upload the checkpoints of the model to Google Cloud Storage: | ||
198 | + | ||
199 | +``` | ||
200 | +gsutil cp ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03/model.ckpt.* gs://YOUR_UNIQUE_BUCKET_NAME/data/ | ||
201 | +``` | ||
202 | + | ||
203 | + | ||
204 | + | ||
205 | +### 3.2. configuring the object detection pipeline | ||
206 | + | ||
207 | + In the TensorFlow Object Detection API, the model parameters, training and eval parameters are defined by a `.config` file. More details in [here](https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/configuring_jobs.md). Anyway, we'll need to configure some paths in order for the template to work. It will be `PATH_TO_BE_CONFIGURED` in the code. Copy and move the config file corresponding to the model received for modification. | ||
208 | + | ||
209 | +``` | ||
210 | +cp YOUR_PATH/models/research/object_detection/samples/configs/ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync.config . | ||
211 | +``` | ||
212 | + | ||
213 | + | ||
214 | + | ||
215 | +First, modify the `PATH_TO_BE_CONFIGURED` of the copied file to our Google Cloud Storage bucket. | ||
216 | + | ||
217 | +``` | ||
218 | +sed -i "s|PATH_TO_BE_CONFIGURED|"gs://YOUR_UNIQUE_BUCKET_NAME"/data|g" ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync.config | ||
219 | +``` | ||
220 | + | ||
221 | + | ||
222 | + | ||
223 | +In addition, modify the other things (`input_path`, `output_path`,`batch_size`, `num_steps`, `total_steps`, and so on). | ||
224 | + | ||
225 | +From: | ||
226 | + | ||
227 | +``` | ||
228 | + . | ||
229 | + . | ||
230 | +train_config: { | ||
231 | + fine_tune_checkpoint: "gs://YOUR_UNIQUE_BUCKET_NAME/data/model.ckpt" | ||
232 | + batch_size: 64 # | ||
233 | + sync_replicas: true | ||
234 | + startup_delay_steps: 0 | ||
235 | + replicas_to_aggregate: 8 | ||
236 | + num_steps: 25000 # | ||
237 | + data_augmentation_options { | ||
238 | + random_horizontal_flip { | ||
239 | + } | ||
240 | + } | ||
241 | + . | ||
242 | + . | ||
243 | +train_input_reader: { | ||
244 | + tf_record_input_reader { | ||
245 | + input_path: "gs://YOUR_UNIQUE_BUCKET_NAME/data/mscoco_train.record-00000-of-00100" # | ||
246 | + } | ||
247 | + label_map_path: "gs://YOUR_UNIQUE_BUCKET_NAME/data/mscoco_label_map.pbtxt" # | ||
248 | +} | ||
249 | + . | ||
250 | + . | ||
251 | +``` | ||
252 | + | ||
253 | + | ||
254 | + | ||
255 | +To: | ||
256 | + | ||
257 | +``` | ||
258 | + . | ||
259 | + . | ||
260 | +train_config: { | ||
261 | + fine_tune_checkpoint: "gs://YOUR_UNIQUE_BUCKET_NAME/data/model.ckpt" | ||
262 | + batch_size: 16 # | ||
263 | + sync_replicas: true | ||
264 | + startup_delay_steps: 0 | ||
265 | + replicas_to_aggregate: 8 | ||
266 | + num_steps: 2000 # | ||
267 | + data_augmentation_options { | ||
268 | + random_horizontal_flip { | ||
269 | + } | ||
270 | + } | ||
271 | + . | ||
272 | + . | ||
273 | +train_input_reader: { | ||
274 | + tf_record_input_reader { | ||
275 | + input_path: "gs://YOUR_UNIQUE_BUCKET_NAME/data/train.record" # | ||
276 | + } | ||
277 | + label_map_path: "gs://YOUR_UNIQUE_BUCKET_NAME/data/LABEL_MAP..pbtxt" # | ||
278 | +} | ||
279 | + . | ||
280 | + . | ||
281 | +``` | ||
282 | + | ||
283 | +This is just an example. Therefore, you should modify the `.config` as you intended. | ||
284 | + | ||
285 | + | ||
286 | + | ||
287 | +Just like any other files, upload the `.config` to your Google Cloud Storage bucket. | ||
288 | + | ||
289 | +``` | ||
290 | +gsutil cp ssd_mobilenet_v1_fpn_shared_box_predictor_640x640_coco14_sync.config gs://YOUR_UNIQUE_BUCKET_NAME/data/pipeline.config | ||
291 | +``` | ||
292 | + | ||
293 | +We're ready now! Let's train. | ||
294 | + | ||
295 | + | ||
296 | + | ||
297 | +## 4. Training and Evaluation Jobs on Google Cloud ML Engine | ||
298 | + | ||
299 | + Before we start our training job in Google Cloud ML Engine, we need to package the API, pycocotools and TF slim. | ||
300 | + | ||
301 | +``` | ||
302 | +# From YOUR_PATH/models/research/ | ||
303 | +bash object_detection/dataset_tools/create_pycocotools_package.sh /tmp/pycocotools | ||
304 | +python setup.py sdist | ||
305 | +(cd slim && python setup.py sdist) | ||
306 | +``` | ||
307 | + | ||
308 | + | ||
309 | + | ||
310 | +And start training! Run following gcloud command: | ||
311 | + | ||
312 | +``` | ||
313 | +gcloud ml-engine jobs submit training `whoami`_object_detection_`date +%s` \ | ||
314 | + --job-dir=gs://YOUR_UNIQUE_BUCKET_NAME/train \ | ||
315 | + --packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz,/tmp/pycocotools/pycocotools-2.0.tar.gz \ | ||
316 | + --module-name object_detection.model_tpu_main \ | ||
317 | + --runtime-version 1.14 \ | ||
318 | + --scale-tier BASIC_TPU \ | ||
319 | + --region us-central1 \ | ||
320 | + -- \ | ||
321 | + --model_dir=gs://YOUR_UNIQUE_BUCKET_NAME/train \ | ||
322 | + --tpu_zone us-central1 \ | ||
323 | + --pipeline_config_path=gs://YOUR_UNIQUE_BUCKET_NAME/data/pipeline.config | ||
324 | +``` | ||
325 | + | ||
326 | + | ||
327 | + | ||
328 | +After we start our training , run following command: | ||
329 | + | ||
330 | +``` | ||
331 | +gcloud ml-engine jobs submit training `whoami`_object_detection_eval_validation_`date +%s` \ | ||
332 | + --job-dir=gs://YOUR_UNIQUE_BUCKET_NAME/train \ | ||
333 | + --packages dist/object_detection-0.1.tar.gz,slim/dist/slim-0.1.tar.gz,/tmp/pycocotools/pycocotools-2.0.tar.gz \ | ||
334 | + --module-name object_detection.model_main \ | ||
335 | + --runtime-version 1.14 \ | ||
336 | + --scale-tier BASIC_GPU \ | ||
337 | + --region us-central1 \ | ||
338 | + -- \ | ||
339 | + --model_dir=gs://YOUR_UNIQUE_BUCKET_NAME/train \ | ||
340 | + --pipeline_config_path=gs://YOUR_UNIQUE_BUCKET_NAME/data/pipeline.config \ | ||
341 | + --checkpoint_dir=gs://YOUR_UNIQUE_BUCKET_NAME/train | ||
342 | +``` | ||
343 | + | ||
344 | + | ||
345 | + | ||
346 | +That's all! You can use TensorBoard to see the accuracy of your model. | ||
347 | + | ||
348 | +``` | ||
349 | +# This command needs to be run once to allow your local machine to access your | ||
350 | +# GCS bucket. | ||
351 | +gcloud auth application-default login | ||
352 | + | ||
353 | +tensorboard --logdir=gs://YOUR_UNIQUE_BUCKET_NAME/model_dir | ||
354 | +``` | ||
355 | + | ||
356 | +![1_4](./imgs/1_4.PNG) | ||
357 | + | ||
358 | + | ||
359 | + |
-
Please register or login to post a comment