List of usage examples for twitter4j Query setCount
public void setCount(int count)
From source file:com.data.dataanalytics.twitter.TwitterFeed.java
public List<Tweet> getTweets(String search) { // We're curious how many tweets, in total, we've retrieved. Note that TWEETS_PER_QUERY is an upper limit, // but Twitter can and often will retrieve far fewer tweets twitter4j.Twitter twitter = getTwitter(); /* This variable is the key to our retrieving multiple blocks of tweets. In each batch of tweets we retrieve, we use this variable to remember the LOWEST tweet ID. Tweet IDs are (java) longs, and they are roughly sequential over time. Without setting the MaxId in the query, Twitter will always retrieve the most recent tweets. Thus, to retrieve a second (or third or ...) batch of Tweets, we need to set the Max Id in the query to be one less than the lowest Tweet ID we've seen already. This allows us to page backwards through time to retrieve additional blocks of tweets*/ long maxID = -1; try {// ww w. j a v a 2s . c o m // There are limits on how fast you can make API calls to Twitter, and if you have hit your limit // and continue to make calls Twitter will get annoyed with you. I've found that going past your // limits now and then doesn't seem to be problematic, but if you have a program that keeps banging // the API when you're not allowed you will eventually get shut down. // // Thus, the proper thing to do is always check your limits BEFORE making a call, and if you have // hit your limits sleeping until you are allowed to make calls again. // // Every time you call the Twitter API, it tells you how many calls you have left, so you don't have // to ask about the next call. But before the first call, we need to find out whether we're already // at our limit. // This returns all the various rate limits in effect for us with the Twitter API Map<String, RateLimitStatus> rateLimitStatus = twitter.getRateLimitStatus("search"); // This finds the rate limit specifically for doing the search API call we use in this program RateLimitStatus searchTweetsRateLimit = rateLimitStatus.get("/search/tweets"); // Always nice to see these things when debugging code... System.out.printf("You have %d calls remaining out of %d, Limit resets in %d seconds\n", searchTweetsRateLimit.getRemaining(), searchTweetsRateLimit.getLimit(), searchTweetsRateLimit.getSecondsUntilReset()); // This is the loop that retrieve multiple blocks of tweets from Twitter for (int queryNumber = 0; queryNumber < MAX_QUERIES; queryNumber++) { System.out.printf("\n\n!!! Starting loop %d\n\n", queryNumber); // Delay if (searchTweetsRateLimit.getRemaining() == 0) { System.out.printf("!!! Sleeping for %d seconds due to rate limits\n", searchTweetsRateLimit.getSecondsUntilReset()); // If you sleep exactly the number of seconds, you can make your query a bit too early // and still get an error for exceeding rate limitations // // Adding two seconds seems to do the trick. Sadly, even just adding one second still triggers a // rate limit exception more often than not. I have no idea why, and I know from a Comp Sci // standpoint this is really bad, but just add in 2 seconds and go about your business. Or else. Thread.sleep((searchTweetsRateLimit.getSecondsUntilReset() + 2) * 1000l); } Query q = new Query(search); // Search for tweets that contains this term q.setCount(TWEETS_PER_QUERY); // How many tweets, max, to retrieve //q.resultType("recent"); // Get all tweets q.setLang("en"); // English language tweets, please // If maxID is -1, then this is our first call and we do not want to tell Twitter what the maximum // tweet id is we want to retrieve. But if it is not -1, then it represents the lowest tweet ID // we've seen, so we want to start at it-1 (if we start at maxID, we would see the lowest tweet // a second time... if (maxID != -1) { q.setMaxId(maxID - 1); } // This actually does the search on Twitter and makes the call across the network QueryResult r = twitter.search(q); // If there are NO tweets in the result set, it is Twitter's way of telling us that there are no // more tweets to be retrieved. Remember that Twitter's search index only contains about a week's // worth of tweets, and uncommon search terms can run out of week before they run out of tweets if (r.getTweets().size() == 0) { break; // Nothing? We must be done } // loop through all the tweets and process them. Need to save as CSV file for database for (Status s : r.getTweets()) { // Loop through all the tweets... // Increment our count of tweets retrieved // Keep track of the lowest tweet ID. If you do not do this, you cannot retrieve multiple // blocks of tweets... if (maxID == -1 || s.getId() < maxID) { maxID = s.getId(); } // Do something with the tweet.... ta.processTweets(s, new Date()); } // As part of what gets returned from Twitter when we make the search API call, we get an updated // status on rate limits. We save this now so at the top of the loop we can decide whether we need // to sleep or not before making the next call. searchTweetsRateLimit = r.getRateLimitStatus(); } } catch (Exception e) { // Catch all -- you're going to read the stack trace and figure out what needs to be done to fix it System.out.println("That didn't work well...wonder why?"); e.printStackTrace(); } System.out.printf("\n\nA total of %d tweets retrieved\n", ta.getTotalTweets()); System.out.println("The total amount of tweets in an hour " + ta.getTweetsInAnHour()); ta.checkIfTrending(ta.getTweetsInAnHour(), ta.getTotalTweets()); return ta.getTweetList(); }
From source file:com.dhamacher.tweetsentimentanalysis.Main.java
License:Open Source License
public static void main(String args[]) { try {/*from w w w .ja v a2 s . c o m*/ license(); /* Prompt for Search token */ Scanner in = new Scanner(System.in); System.out.print("Type in the search query: "); Query searchToken = new Query(in.nextLine()); /* Set maximum count */ searchToken.setCount(1500); /* Send search request to Twitter and receive List<> of results */ QueryResult result = twitter.search(searchToken); /* Notify user of the result */ System.out.println( "Amount of tweets pulled for \"" + searchToken.getQuery() + "\": " + result.getTweets().size()); persistTweet(result, searchToken.getQuery()); end(); } catch (TwitterException ex) { Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex); } }
From source file:com.fsatir.twitter.TwitterManagedBean.java
public List<Media> trendImageList(Twitter twitter, String trendName) { Query query = new Query(trendName + " AND filter:images"); query.setCount(1); QueryResult result;/*from w w w. ja v a2s . com*/ try { do { result = twitter.search(query); List<twitter4j.Status> tweets = result.getTweets(); int counter = 0; for (twitter4j.Status status : tweets) { Media myMedia = new Media(); int control = 0; for (MediaEntity me : status.getMediaEntities()) { //Tek grsel almay kesinletirmek iin kontrol. //leride modeldeki trendImgURL diziye dntrlerek kontrol kaldrlabilir. if (control < 1) myMedia.setTrendImgURL(me.getMediaURLHttps()); control++; } // URL null deilse, grsel ve bigileri list'e eklenir. if (myMedia.getTrendImgURL() != null) { myMedia.setTweetID(status.getId()); myMedia.setFavorite_count(status.getFavoriteCount()); myMedia.setRetweet_count(status.getRetweetCount()); //myMedia.setTrendImgURL(status.getMediaEntities()[0].getMediaURLHttps()); myMedia.setTrendName(trendName); myMedia.setName(Long.toString(status.getId())); myMedia.setType(status.getMediaEntities()[0].getType()); myMediaList.add(counter++, myMedia); } } } while ((query = result.nextQuery()) != null && result.getRateLimitStatus().getRemaining() > 0); } catch (TwitterException e) { e.printStackTrace(); } myMediaListFiltered = filterList(myMediaList); return myMediaListFiltered; }
From source file:com.fuzuapp.model.resultados.TwitterAdapter.java
@Override public List<Resultado> getResultados(GeoPoint ponto, double raio) { List<Resultado> resultados = new ArrayList(); try {// w w w . j a va2 s. c o m Query query = new Query(""); GeoLocation geo = new GeoLocation(ponto.getLatitude(), ponto.getLongitude()); query.setGeoCode(geo, raio, Query.KILOMETERS); query.resultType(Query.RECENT); query.setCount(20); QueryResult result = twitter.search(query); for (Status status : result.getTweets()) { Resultado r = new Resultado(); r.setDescricao(status.getText()); r.setUrl("http://twitter.com/statuses/" + String.valueOf(status.getId())); r.setNomeUsuario(status.getUser().getName()); r.setHorario(new SimpleDateFormat("dd/MM HH:mm").format(status.getCreatedAt())); r.setFotoUrl(status.getUser().getProfileImageURL()); //r.setLocal(new GeoPoint(status.getGeoLocation().getLatitude(), status.getGeoLocation().getLongitude())); r.setTipo(Resultado.TEXTO); resultados.add(r); } } catch (TwitterException ex) { Logger.getLogger(TwitterAdapter.class.getName()).log(Level.SEVERE, null, ex); } return resultados; }
From source file:com.javielinux.database.EntitySearch.java
License:Apache License
public Query getQuery(Context cnt) { String q = this.getString("words_and"); if (!this.getString("words_or").equals("")) { q += Utils.getQuotedText(this.getString("words_or"), "OR ", false); }//from w ww .j a v a2 s. co m if (!this.getString("words_not").equals("")) { q += Utils.getQuotedText(this.getString("words_not"), "-", true); } if (!this.getString("from_user").equals("")) { q += " from:" + this.getString("from_user"); } if (!this.getString("to_user").equals("")) { q += " to:" + this.getString("to_user"); } if (!this.getString("source").equals("")) { q += " source:" + this.getString("source"); } if (this.getInt("attitude") == 1) q += " :)"; if (this.getInt("attitude") == 2) q += " :("; String modLinks = "filter:links"; String websVideos = "twitvid OR youtube OR vimeo OR youtu.be"; String webPhotos = "lightbox.com OR mytubo.net OR imgur.com OR instagr.am OR twitpic OR yfrog OR plixi OR twitgoo OR img.ly OR picplz OR lockerz"; if (this.getInt("filter") == 1) q += " " + modLinks; if (this.getInt("filter") == 2) q += " " + webPhotos + " " + modLinks; if (this.getInt("filter") == 3) q += " " + websVideos + " " + modLinks; if (this.getInt("filter") == 4) q += " " + websVideos + " OR " + webPhotos + " " + modLinks; if (this.getInt("filter") == 5) q += " source:twitterfeed " + modLinks; if (this.getInt("filter") == 6) q += " ?"; if (this.getInt("filter") == 7) q += " market.android.com OR androidzoom.com OR androlib.com OR appbrain.com OR bubiloop.com OR yaam.mobi OR slideme.org " + modLinks; Log.d(Utils.TAG, "Buscando: " + q); Query query = new Query(q); if (this.getInt("use_geo") == 1) { if (this.getInt("type_geo") == 0) { // coordenadas del mapa GeoLocation gl = new GeoLocation(this.getDouble("latitude"), this.getDouble("longitude")); String unit = Query.KILOMETERS; if (this.getInt("type_distance") == 0) unit = Query.MILES; query.setGeoCode(gl, this.getDouble("distance"), unit); } if (this.getInt("type_geo") == 1) { // coordenadas del gps Location loc = LocationUtils.getLastLocation(cnt); if (loc != null) { GeoLocation gl = new GeoLocation(loc.getLatitude(), loc.getLongitude()); String unit = Query.KILOMETERS; if (this.getInt("type_distance") == 0) unit = Query.MILES; query.setGeoCode(gl, this.getDouble("distance"), unit); } else { mErrorLastQuery = cnt.getString(R.string.no_location); } } } PreferenceManager.setDefaultValues(cnt, R.xml.preferences, false); SharedPreferences preference = PreferenceManager.getDefaultSharedPreferences(cnt); int count = Integer.parseInt(preference.getString("prf_n_max_download", "60")); if (count <= 0) count = 60; query.setCount(count); String lang = ""; if (!this.getString("lang").equals("")) lang = this.getString("lang"); if (!lang.equals("all")) query.setLang(lang); // obtener desde donde quiero hacer la consulta if (getInt("notifications") == 1) { String where = "search_id = " + this.getId() + " AND favorite = 0"; int nResult = DataFramework.getInstance().getEntityListCount("tweets", where); if (nResult > 0) { long mLastIdNotification = DataFramework.getInstance().getTopEntity("tweets", where, "date desc") .getLong("tweet_id"); query.setSinceId(mLastIdNotification); } } //query.setResultType(Query.POPULAR); return query; }
From source file:com.mycompany.omnomtweets.TweetsAboutCandidates.java
/** * Searches for tweets using the given query string. * @param str the query to use// ww w . ja v a2 s. c o m * @return List of the tweet statuses that match the query. */ public List<Status> search(String str, long maxId) { List<Status> tweets = null; try { Query query = new Query(str); query.setCount(100); //English only. query.setLang("en"); QueryResult result; query.setMaxId(maxId); result = twitter.search(query); tweets = result.getTweets(); } catch (TwitterException te) { System.out.println("Failed to search tweets: " + te.getMessage()); } return tweets; }
From source file:com.mycompany.twitterproductanalysistool.TwitterAPI.java
public ArrayList<String> getQuery(String qry) { tweets = new ArrayList<>(); tweetCountries = new ArrayList<>(); tweetDates = new ArrayList<>(); try {/* w w w. j av a 2 s . c om*/ Query query = new Query(qry); query.setCount(99); result = twitter.search(query); for (Status status : result.getTweets()) { tweets.add(status.getText()); tweetDates.add(status.getCreatedAt()); if (status.getUser() == null) { tweetCountries.add("NO COUNTRY"); } else { tweetCountries.add(status.getUser().getLocation()); } } } catch (TwitterException te) { te.printStackTrace(); System.out.println("Failed to search tweets: " + te.getMessage()); System.exit(-1); } for (int i = 0; i < tweets.size(); i++) { String s = tweets.get(i); tweets.set(i, s.replace("\n", "")); } return tweets; }
From source file:com.twitter.TwitterCrawler.java
/** * * @return @throws TwitterException/*from w w w .java 2s .c o m*/ * @throws InterruptedException returns the number of tweets */ public TwitterApiResponse stream() throws TwitterException, InterruptedException { //************************ Variables ************************* TwitterApiResponse response = new TwitterApiResponse(); long timeNow = System.currentTimeMillis(); long afterInterval = timeNow + (interval * 1000); int countTweets = 0; //define the tweet id that the application should read relative to the IDs of Tweets it has already processed long maxID = 0; ArrayList<Status> tweets = new ArrayList<Status>(); Query query = new Query(keywords); //************************ Action ************************* //max 450 calls in 15 mins -->30/min, 5 calls/10 secs while (timeNow < afterInterval) { try { if (sinceID != 0) { query.setSinceId(sinceID); } if (maxID != 0) { query.setMaxId(maxID); } query.setResultType(Query.ResultType.recent); query.setCount(100); QueryResult result = twitter.search(query); tweets.addAll(result.getTweets()); if (tweets.size() == 100) { countTweets += tweets.size(); maxID = tweets.get(tweets.size() - 1).getId() - 1; } else { countTweets += tweets.size(); break; } timeNow = System.currentTimeMillis(); } catch (TwitterException te) { System.out.println("Couldn't connect: " + te); } } sinceID = tweets.get(0).getId(); response.setCount(countTweets); response.setSinceID(sinceID); return response; }
From source file:crawler.DataSearch.java
License:Apache License
public void collectNewTweets() { while (keywords.size() > 0) { String hash = keywords.get(0); Query query = new Query(hash); query.setCount(Settings.searchTweetNo); QueryResult result;/*www . ja v a 2s . com*/ try { result = twitter.search(query); List<Status> tweets = result.getTweets(); if (tweets.size() == 0) TwitterCrawler.TFFreq.remove(hash); for (Status status : tweets) { //System.out.println("{"+hash+"}["+status.getCreatedAt().toString()+"]- [USER: " + status.getUser().getScreenName() + "] - " + status.getText()); String temp; if (trash(status.getText())) { if (Settings.limitTime) { long currentTime = System.currentTimeMillis(); long tweetTime = status.getCreatedAt().getTime(); if (currentTime - tweetTime > Settings.timeLimit) continue; } if (hashtext.containsKey(hash)) temp = hashtext.get(hash) + status.getText(); else temp = status.getText(); hashtext.put(hash, temp); } } //System.out.println("{"+hash+"} with "+tweets.size()+"tweets"); keywords.remove(0); } catch (TwitterException e) { if (e.getErrorCode() == 88) { System.out.println("Using other keys???"); String ConsumerKey_temp = Settings.ConsumerKey.get(0); String ConsumerSecret_temp = Settings.ConsumerSecret.get(0); String AccessToken_temp = Settings.AccessToken.get(0); String AccessSecret_temp = Settings.AccessSecret.get(0); Settings.ConsumerKey.remove(ConsumerKey_temp); Settings.ConsumerSecret.remove(ConsumerSecret_temp); Settings.AccessToken.remove(AccessToken_temp); Settings.AccessSecret.remove(AccessSecret_temp); twitter = new TwitterFactory( OA.build(Settings.ConsumerKey.get(0), Settings.ConsumerSecret.get(0), Settings.AccessToken.get(0), Settings.AccessSecret.get(0))).getInstance(); Settings.ConsumerKey.add(Settings.ConsumerKey.size(), ConsumerKey_temp); Settings.ConsumerSecret.add(Settings.ConsumerSecret.size(), ConsumerSecret_temp); Settings.AccessToken.add(Settings.AccessToken.size(), AccessToken_temp); Settings.AccessSecret.add(Settings.AccessSecret.size(), AccessSecret_temp); System.out.println("**************************************************************"); System.out.println("[" + ConsumerKey_temp + "] to [" + Settings.ConsumerKey.get(0) + "]"); System.out.println("[" + ConsumerSecret_temp + "] to [" + Settings.ConsumerSecret.get(0) + "]"); System.out.println("[" + AccessToken_temp + "] to [" + Settings.AccessToken.get(0) + "]"); System.out.println("[" + AccessSecret_temp + "] to [" + Settings.AccessSecret.get(0) + "]"); System.out.println("**************************************************************"); continue; } e.printStackTrace(); } } }
From source file:crawler.TwitterFeed.java
/** * *///from w w w. j a v a 2 s. c o m public void get_tweets() throws SQLException { CSVReader cr = new CSVReader(); ArrayList<String> names = new ArrayList<>(); Map<String, String> name_ticker; names = cr.get_company_names(tickers); System.out.println(names); ConfigurationBuilder cb = new ConfigurationBuilder(); cb.setOAuthConsumerKey("WNSycI2GS33ZAwHJ9Fb4A"); cb.setOAuthConsumerSecret("BBWeSThT3ZV2g9c5BDuUduDyNAWyzouMI0XjQy7KUc"); cb.setOAuthAccessToken("1852271029-SLfE061bImfcRxWQZpy1pAgpEkfFhg3757Q9JRf"); cb.setOAuthAccessTokenSecret("1uPSfYPbaENtXFYWsryBIAHnUsmG3pT6lGH0NzxTCGW7O"); Twitter twitter = new TwitterFactory(cb.build()).getInstance(); for (String s : names) { String[] temp; if (s.contains(",")) { temp = s.split(","); } else { temp = s.split(" "); } System.out.println( "--------------------------------------------------------------------------------------" + temp[0]); Query query = new Query("#" + temp[0]); query.lang("en"); int numberOfTweets = 500; long lastID = Long.MAX_VALUE; ArrayList<Status> tweets = new ArrayList<>(); while (tweets.size() < numberOfTweets) { if (numberOfTweets - tweets.size() > 100) { query.setCount(100); } else { query.setCount(numberOfTweets - tweets.size()); } try { QueryResult result = twitter.search(query); tweets.addAll(result.getTweets()); System.out.println("Gathered " + tweets.size() + " tweets"); for (Status t : tweets) { if (t.getId() < lastID) { lastID = t.getId(); } } } catch (TwitterException te) { System.out.println("Couldn't connect: " + te); } query.setMaxId(lastID - 1); } Connection con = DriverManager.getConnection("jdbc:mysql://localhost:3306/CrawledData", "root", ""); for (int i = 0; i < tweets.size(); i++) { Status t = (Status) tweets.get(i); //GeoLocation loc = t.getGeoLocation(); String user = t.getUser().getScreenName(); String msg = t.getText(); String time = t.getCreatedAt().toString(); //if (loc!=null) { // Double lat = t.getGeoLocation().getLatitude(); // Double lon = t.getGeoLocation().getLongitude(); // System.out.println(i + " USER: " + user + " wrote: " + msg + " located at " + lat + ", " + lon); //} //else{ System.out.println(i + " USER: " + user + " wrote: " + msg + " at: " + time); //} try { Class.forName("com.mysql.jdbc.Driver"); PreparedStatement ps = con .prepareStatement("INSERT IGNORE INTO TwitterFeed VALUES (?,?,?,?,?)"); ps.setString(1, s); ps.setString(2, cr.get_ticker_from_company(s)); ps.setString(3, user); ps.setString(4, msg); ps.setString(5, time); int k = ps.executeUpdate(); if (k <= 0) { System.out.println("Entry Unsuccessful"); } else { System.out.println("Entry Successful"); } } catch (ClassNotFoundException | SQLException e) { System.out.println(e); } } con.close(); } }