Day 2 - BYU-CS260-Winter-2018/redbird GitHub Wiki
For Day 2, we will add search to our Twitter clone. We'll also add some testing data so we can test our search, plus we'll convert embedded URLs and hashtags to links.
Our first step is to create a database migration. To speed up searching, we will add an index to our tweets table. Start by creating the migration:
npx knex migrate:make search
Then edit the migration file this creates and add the following:
exports.up = function(knex, Promise) {
return Promise.all([
knex.raw("alter table tweets add fulltext(tweet)"),
]);
};
exports.down = function(knex, Promise) {
return Promise.all([
knex.raw("alter table tweets drop index tweet"),
]);
};
This adds a full text index on the tweet column inside the tweets table. You can see the MariaDB Documentation for Fulltext Search for details.
We'll then apply this migration:
npx knex migrate:latest
We now need to add a REST API endpoint for searching tweets. Our endpoint will accept URLs of the form:
/api/tweets/search?keywords=javascript&limit=10&offset=20
This asks the server to search using the keyword "javascript", to limit the results to 10 tweets, and to start the results at the 20th one that is returned. You can use a generic search endpoint like this to implement either a paged search result (like Google search) or an "infinite scroll" search that loads new results each time you scroll to the bottom of the page.
Edit server.js and add this:
app.get('/api/tweets/search', (req, res) => {
if (!req.query.keywords)
return res.status(400).send();
let offset = 0;
if (req.query.offset)
offset = parseInt(req.query.offset);
let limit = 50;
if (req.query.limit)
limit = parseInt(req.query.limit);
knex('users').join('tweets','users.id','tweets.user_id')
.whereRaw("MATCH (tweet) AGAINST('" + req.query.keywords + "')")
.orderBy('created','desc')
.limit(limit)
.offset(offset)
.select('tweet','username','name','created').then(tweets => {
res.status(200).json({tweets:tweets});
}).catch(error => {
res.status(500).json({ error });
});
});
This query takes three parameters, but only the search keywords are mandatory. We provide default values for the offset and limit.
To perform the query, we join the users and tweets tables together, so we can return the details about the user who created each tweet. We then use a raw where clause. This is a useful feature of knex -- it lets us provide raw SQL, either to handle cases where knex doesn't have a wrapper for some functionality, or just for personal preference. We then add orderBy, limit, and offset to the query.
Our next step is to setup the store to query the REST API. Update store.js with this method:
+ doSearch(context,keywords) {
+ axios.get("/api/tweets/search?keywords=" + keywords).then(response => {
+ context.commit('setFeed',response.data.tweets);
+ }).catch(err => {
+ console.log("doSearch failed:",err);
+ });
+ },
Most of our changes, surprisingly, are on the front end, because we'll need to do some refactoring to improve our code.
First, modify index.html to load the Font Awesome icon library. We'll use this to provide a search icon in a search menu.
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1.0">
+ <script defer src="https://use.fontawesome.com/releases/v5.0.9/js/all.js" integrity="sha384-8iPTk2s/jMVj81dnzb/iFR2sdA7u06vHJyyLlAd4snFpCl/SnyUjRrbdJsw1pGIl" crossorigin="anonymous"></script>
<title>Red Bird</title>
</head>
Next, update some styles in src/App.vue. This first one is not strictly necessary, but it makes our buttons a little more streamlined.
src/App.vue:
border: none;
border-radius: 5px;
color: white;
- padding: 10px;
+ padding: 5px;
text-align: center;
text-decoration: none;
display: inline-block;
- font-size: 1.2rem;
+ font-size: 1rem;
margin: 4px 2px;
}
This one changes the color of our links to match our color scheme. The purple was getting kind of annoying.
height: 20px;
padding: 10px;
}
+ a {
+ color: #77C4D3;
+ }
.flexWrapper {
display:flex;
}
We're going to put our search form inside of the menu. To do this, modify src/components/AppHeader.vue:
<ul id="menu">
<li><img src="/static/images/red-bird.png"/></li>
<li><router-link to="/">Home</router-link></li>
+ <li><form v-on:submit.prevent="search">
+ <input v-model="keywords" placeholder="Search">
+ <a href="#" v-on:click="search" class="search"><i class="fas fa-search"></i></a>
+ </form></li>
<li class="right" v-if="loggedIn"><a @click="logout" href="#">Logout</a></li>
<li class="right" v-if="loggedIn">{{user.username}}</li>
- <form v-else class="right" v-on:submit.prevent="login">
- <input v-model="email" placeholder="Email Address">
- <input v-model="password" placeholder="Password">
- <button class="primary" type="submit">Login</button>
- </form>
+ <li class="right" v-else>
+ <form v-on:submit.prevent="login">
+ <input v-model="email" placeholder="Email Address">
+ <input v-model="password" type="password" placeholder="Password">
+ <button class="primary" type="submit">Login</button>
+ </form>
+ </li>
</ul>
In addition to the new search menu, we updated the login menu to display properly inside a list item and to hide the password when we type it.
We also want to change the data properties of this component to keep track of the keywords typed in the search box:
data () {
return {
+ keywords: '',
email: '',
password: '',
}
And we need to add a method to do the search:
methods: {
+ search: function() {
+ this.$router.push({ path: '/search', query: { keywords: this.keywords }})
+ this.keywords = '';
+ },
Notice that entering a search in the form will trigger what is called a programmatic navigation -- meaning we manually make the change to a new route inside of Vue. We'll navigate to a search results page.
We also need to add a few styles:
+ input {
+ height: 0.5em;
+ }
+ .search {
+ margin-left: 5px;
+ }
That's it for our search form in the menu!
Next, we need to add a component for the search results page. Add a new file, src/components/SearchResults.vue. Here is the template:
<template>
<div class="feed">
<h1>Search Results</h1>
<p>Searched for {{keywords}}</p>
<feed-list/>
</div>
</template>
This displays just a header and the keywords that the user searched for, then includes the template from a component we will define next.
Here is the JavaScript:
<script>
import FeedList from './FeedList';
export default {
name: 'SearchResults',
components: { FeedList },
created: function() {
this.$store.dispatch('doSearch',this.$route.query.keywords);
},
computed: {
keywords: function() {
return this.$route.query.keywords;
}
},
watch: {
'$route.query.keywords'() {
this.$store.dispatch('doSearch',this.$route.query.keywords);
}
},
}
</script>
We use a created hook to do the search. We also have to use a watch function to repeat the search if the query keywords ever change.
The search results are displayed in a separate component, called FeedList. Create this component in src/components/Feedlist.vue.
Here is the template:
<template>
<div>
<div v-for="item in feed" class="item">
<p class="idline"><span class="user">{{item.name}}</span><span class="handle">@{{item.username}}</span><span class="time">{{item.created | since}}</span></p>
<p class="tweet">{{item.tweet}}</p>
</div>
</div>
</template>
This loops through the tweets in the feed to display them.
Here is the JavaScript:
<script>
import moment from 'moment';
export default {
name: 'FeedList',
filters: {
since: function(datetime) {
moment.locale('en', {
relativeTime: {
future: 'in %s',
past: '%s',
s: 'seconds',
ss: '%ss',
m: '1m',
mm: '%dm',
h: 'h',
hh: '%dh',
d: 'd',
dd: '%dd',
M: ' month',
MM: '%dM',
y: 'a year',
yy: '%dY'
}
});
return moment(datetime).fromNow();
},
},
computed: {
feed: function() {
return this.$store.getters.feed;
},
},
}
</script>
This should look really familiar! We're using the same code that is in the UserFeed component to get and display the tweets.
Here is the CSS:
<style scoped>
.item {
border-bottom: 1px solid #ddd;
padding: 10px;
}
.tweet {
margin-top: 0px;
}
.idline {
margin-bottom: 0px;
}
.user {
font-weight: bold;
margin-right: 10px;
}
.handle {
margin-right: 10px;
color: #666;
}
.time {
float: right;
color: #666;
}
</style>
You may have noticed we copied the code for the FeedList component from UserFeed. There's no reason to put this same code in two places, so we will refactor UserFeed to also use FeedList. We'll need to strip out the stuff FeedList does, and leave behind just what is unique to the UserFeed.
Here is the template:
<template>
<div class="feed">
<div>
<form v-on:submit.prevent="tweet" class="tweetForm">
<textarea v-model="text" placeholder=""/><br/>
<div class="buttonWrap">
<button class="primary" type="submit">Tweet</button>
</div>
</form>
</div>
<feed-list/>
</div>
</template>
We still need the form to create a new tweet, but just include the FeedList after that.
Here is the JavaScript:
<script>
import FeedList from './FeedList';
export default {
name: 'UserFeed',
components: { FeedList },
data () {
return {
text: '',
}
},
created: function() {
this.$store.dispatch('getFeed');
},
methods: {
tweet: function() {
this.$store.dispatch('addTweet',{
tweet: this.text,
}).then(tweet => {
this.text = "";
});
},
}
}
</script>
Here we need everything related to the form that creates a tweet, so this is just the data property and the method for submitting the form.
Finally, here are the CSS styles that pertain to the form:
<style scoped>
.feed {
width: 600px;
}
.tweetForm {
background: #eee;
padding: 10px;
margin-bottom: 10px;
}
.buttonWrap {
width: 100%;
display: flex;
}
button {
margin-left: auto;
height: 2em;
font-size: 0.9em;
}
textarea {
width: 100%;
height: 5em;
padding: 2px;
margin-bottom: 5px;
resize: none;
box-sizing: border-box;
}
</style>
We also need to update the routing in src/router/index.js:
import Vue from 'vue'
import Router from 'vue-router'
import HomePage from '@/components/HomePage'
+import SearchResults from '@/components/SearchResults'
Vue.use(Router)
and
name: 'HomePage',
component: HomePage
},
+ {
+ path: '/search',
+ name: 'SearchResults',
+ component: SearchResults
+ },
+
At this point, you should be able to test search in your code and see it working. But you'll have to create an account, login, and create some tweets you can search on.
To make testing easier, we'll insert some test data in the database. Create a directory called data. In this directory, put this file:
Warning: Running this file will delete all the tweets and users in your database and replace them with test data that is scraped from Twitter.
Run this with:
node data/addTweets.js
You can now try searching over this data. Hint, use the keyword "javascript".
You can also login as any of these test users. Just add @test.com to their username and use test as the password.
For example, you can login as [email protected] with the password test.
Now that we have search, we would like to be able to automatically link hash tags to search. We can also automatically convert URLs into links.
Searching on hashtags needs a different SQL query. So we'll provide a new REST API endpoint in server.js, using the
URL /api/tweets/hash/:
app.get('/api/tweets/hash/:hashtag', (req, res) => {
let offset = 0;
if (req.query.offset)
offset = parseInt(req.query.offset);
let limit = 50;
if (req.query.limit)
limit = parseInt(req.query.limit);
knex('users').join('tweets','users.id','tweets.user_id')
.whereRaw("tweet REGEXP '^#" + req.params.hashtag + "' OR tweet REGEXP ' #" + req.params.hashtag + "'")
.orderBy('created','desc')
.limit(limit)
.offset(offset)
.select('tweet','username','name','created').then(tweets => {
res.status(200).json({tweets:tweets});
}).catch(error => {
res.status(500).json({ error });
});
});
Notice that for searching on hashtags we need to use regular expressions to be sure the pound symbol is at the start of the word.
Next we need to create an action in the store to search this endpoint. Add a new method in src/store.js:
doHashTagSearch(context,hashtag) {
axios.get("/api/tweets/hash/" + hashtag).then(response => {
context.commit('setFeed',response.data.tweets);
}).catch(err => {
console.log("doHashTagSearch failed:",err);
});
}
Then we need to create a new component in src/components/HashTag.vue. This component is very similar
to UserFeed.vue. Here is the template, which just shows the header for the search results, and then uses
FeedList.vue to show the search results:
<template>
<div class="feed">
<h1>Hashtag Results</h1>
<p>Searched for #{{hashtag}}</p>
<feed-list/>
</div>
</template>
Here is the JavaScript:
<script>
import FeedList from './FeedList';
export default {
name: 'HashTag',
components: { FeedList },
created: function() {
this.$store.dispatch('doHashTagSearch',this.$route.params.hashtag);
},
computed: {
hashtag: function() {
return this.$route.params.hashtag;
}
},
watch: {
'$route.params.hashtag'() {
this.$store.dispatch('doHashTagSearch',this.$route.params.hashtag);
}
},
}
</script>
We dispatch to the store when we create the component or if the hashtag search changes. We use a computed function to make the hashtag available to the template.
We need to add a new routing endpoint to handle this component. Update src/router/index.js:
import Vue from 'vue'
import Router from 'vue-router'
import HomePage from '@/components/HomePage'
import SearchResults from '@/components/SearchResults'
+import HashTag from '@/components/HashTag'
Vue.use(Router)
and
name: 'SearchResults',
component: SearchResults
},
+ {
+ path: '/hashtag/:hashtag',
+ name: 'HashTag',
+ component: HashTag
+ },
+
While we are in the router, also add this:
export default new Router({
+ mode: 'history',
routes: [
This tells the Vue Router not to use the "#" at the start of routes, like it normally does.
We need to scan tweets for hashtags and URLs and then turn these into links. Because of this, we need to be sure not to allow other HTML inside of tweets (e.g. loading a malicious script). To prevent this, we use HTML escaping.
Download this file for HTML escaping and put it in src/components/htmlEscape.js:
Download this file for converting URLs and hashtags into links and put it in src/components/linkify.js:
Now we need to call these functions whenever we show a tweet. Because we refactored our code, this is fairly easy to do. Modify the FeedList component. First, change the way tweets are displayed so that they are formatted with a function:
<div>
<div v-for="item in feed" class="item">
<p class="idline"><span class="user">{{item.name}}</span><span class="handle">@{{item.username}}</span><span class="time">{{item.created | since}}</span></p>
- <p class="tweet">{{item.tweet}}</p>
+ <p v-html="formatTweet(item.tweet)" class="tweet"></p>
</div>
</div>
</template>
We need to import the library:
import moment from 'moment';
+ import linkify from './linkify.js';
export default {
And define a method to call the linkify function:
+ methods: {
+ formatTweet: function(text) {
+ return linkify(text, {
+ defaultProtocol: 'https'
+ });
+ },
+ },
You should now be able to search based on hash tags.