With the emergence of some Deep neural networks(DNN), such as ResNet, the ImageNet challenge is not so difficult as before. As a result, since recognition tasks are going to be solved, we are going to deal with a high-level question, including visual understanding.
Visual Question Answering has attracted much attention these years. Besides, visual relation is also going to be an issue we should consider.
more >>
What is attention?
In fact, the concept of “attention” in machine learning is just the same as what it is in our daily life. That’s maybe confusing. Imagine: parents always want their kids to concentrate, to focus on one thing they are doing, that’s the so-called “attention”. In other words, when we are reading a fiction story for example, sometimes we might can’t understand some parts. So we are likely to focus on this part more than usual, that’s also “attention”. However, before this concept, what people usually did in both language translation and recognition is just put the whole sentence or picture into the input and get the prediction.
more >>
As the last post said, Professor Garrison wanted us to work on some medical dataset, the camelyon16 was the one we should work on, which is a quite large and difficult one. Somehow, we switch the database to this one ChestX-ray8, a new significant work in CVPR 2017.
more >>
We just got a homework from CSE 190 class, which wants us to slove the camelyon16 contest. To be honest, i think it is a difficult task since i have no knowledge about this before. So i am gonna check things out to see waht we can do!
They present a framework to detect and localize tumors as small as 100 x 100 pixels in Gigapixel microscopy images sized 100000x100000 pixels, leveraging a CNN architecure and obtaining state-of-art results on the Camelyon 16 dataset. At 8 false positives per image, we detect 92.4% of the tumors, relative to 82.7% by the previous best automated approach, where human expert can only achieve 73.2%. AUC above 97%, even discovered 2 slides that were erroneously labeled normal. In a word, i think their approach could considerably reduce false negative rates in Metastases detection
more >>
This passage propose a model that uses the insight that combines objects and predicates to train visual models for models and predicates individually and later combines them together. Additionally, it localizes the objects in the predicated relationships as bounding boxes in the image.
While we poses similar challenges as object detection, one critical difference is that the size of semantic space of relationships is much larger. A fundamental challenge in visual relationship detection is learning from very few examples
Visual Phrases :using 13 common relationships, needs to train O(N^2K) . person-jumping-off-bikesis rare while person and bikes are usual. So the authors propose a visual appearance module that fuses objects and predicates together to jointly predicate, only need O(N + K)
Word vector embeddings naturally similar objects in linking their relationships (e.g., “person riding a horse” and “person riding a elephant”). There is a language module that uses cast pre-trained word vectors to cast relationships into a vector space where similar relationships are optimized to be close to each other.(can even enable zero-shot learning)
“Person ride bike “ offers a comprehensive understanding of an image, connecting CV and NLP. Due to the challenging complexity of finding the relation, this passage offers Visual Translation Embedding network(VTranse: subject+predicate=object) for visual relation detection.(competitive to Lu’s multi-modal model with language priors–[])
Note:
- a novel feature extraction layer that enables object-relation knowledge transfer in a fully-convolutional fashion that supports training and inference in a single forward/backward pass.
This are lots of efforts that connecting computer vision and natural language: visual caption and question answering (mostly connects CNN and RNN,operimized on specialized datasets for specific tasks : image caption or image QA….), falling short in understanding relationships
Assume we have N objects and R relation
the fundamental challenge is O(N^2R).A common solution: to learn separate models for objects and predicates: O(N+R)(even more challenge ,compare “ride bike“ to “ride elephant“). TransE : representing large-scale in lower dimensional space, the relation triplet can be interpreted as a vector translation : person + ride = bike , only need to learn the ride translation vector in the relation space.
For example, person and bike detection serves as the context for ride prediction, Specifically, we propose a novel feature extraction layer that extracts three types of object features used in VTransE: classme,locations and RoI visual features. It uses bilinear feature interpolation[15,20] instead of RoI pooling for differentiable coordinates
more >>Summary:
VTransE, a convolutional network that detects objects and relations, first end-to-end relation detection
A novel visual relation detection learning model for VTansE that incorporates translation embedding and knowledge transfer
VTransE outperforms several strong baselines
Ng大神等人提出的Spectral CLustering非常有意思，证明看得我很惊…
巧妙地利用了矩阵的特征，实现了降维，这是一个启发，如何应用好那些简单基础的数学模型
虽然现在paper里的模型越来越复杂，但我们是不是可以换个角度来思考，从最基础的层面重新想一想？
可惜数学基础还不够好orz 233,这篇就当是补一下线代的知识，顺便证明下这个谱聚类
当然，我觉得这个证明中最重要的，是运用矩阵特征值这一点，值得学一蛤
more >>
这是我的第一篇博文(拖了很久才放上来)
主要内容是关于Clustering中最基本的一个k-means以及相关的几个变种
算是个人学习过程中一个笔记(以及一个的讲稿)，因此有些地方可能写的不是很清楚,下次改正！
more >>
