| com.sun.portal.providers.ProfileProviderAdapter
 com.sun.portal.providers.urlscraper.URLScraperProvider
All known Subclasses: com.sun.portal.providers.xml.XMLProvider,
| URLScraperProvider | | public class URLScraperProvider extends ProfileProviderAdapter implements ProviderProperties(Code) | | A URLScraperProvider is a content provider that can retrieve and display
content from a given URL.
URLScraperProvider acts as an HTTP client and makes a request
for the content of the specified URL and then displays it in the
channel.
Each URLScraper channel has its own timeout attribute. The
channel will wait up to its individual timeout to receive
content.
Forwarding of cookies
Each URLScraper channel has a cookiesToForwardList attribute
that can be set on the in the display profile. If
a cookie is allowed by this attribute, a cookie in the request
coming from the browser will be forwarded to the web server specified
for the URL. allCookies attribute can be set to true to allow
all the cookies. A set-cookie request from that web server
will be sent back to the browser. The set-cookie request
is modified so that the cookie is only sent back to the portal server.
URL Rewriting
The content gathered by the channel will be rewritten if
the rewriter is available. The ruleset used by the rewriter can be
specified in the display profile attribute rulesetID.
Relative URLs are converted to absolute URLs. For example, if your portal server is
http://portal.iplanet.com/ and the web server specified in the
URL is http://foo.sesta.com/ and the file contains
<IMG SRC="/images/blah.gif">
then the content sent back to browser via portal server will be
rewritten as:
<IMG SRC="http://foo.sesta.com/images/blah.gif">
Because otherwise the browser will attempt to read the image from
http://portal.sesta.com/images/blah.gif and will not resolve it.
SSL protected pages
In general the URLScraperProvider will work with SSL pages. The
important thing to remember is that there can be no level of
interaction required by the specified URL as there is no way to
pass that information to the end user.
- The certificate on the target server needs to be valid (in other words
signed by a recognized CA) If you browse to the specified URL
and get a warning dialog about an expired or unrecognized certificate,
the URLScraperProvider will not be able to access the site because
it cannot respond to the dialogs.
- There may be difficulties with sites that require 128-bit SSL.
Timeouts
There are 2 timeout values to consider:
- URLScraper timeout (specific to the channel)
- RefreshTime (attribute for all channels)
Each URLScraper channel has its own timeout attribute. The
channel will wait up to its individual timeout to receive
content.
Encoding
The order for determining the encoding would be
HTTP header, if available (only applies to http(s) urls)
inputEncoding property, if non-blank
tag in content, e.g. meta tag in html & wml, xml header for xml, if available
(only applies to HTML, XML,WML determined based on the MIMEType)
system default
MIMEType is determined from the jvm table. If not set, it is determined
from the file extension.
Proxy Configuration
URLScraper channel uses a proxy to scrape the url specified
if the proxy is set in jvm12.conf file for web server
For Example the proxy can be set as
http.proxyHost=
http.proxyPort=
The refreshTime attribute is used for caching and
will cause the URL not to be fetched again if the page is reloaded
within that time.
|
| typeTable | | protected static String typeTable(Code) | | Array of File extensions mapped to the MIMETypes
|
| URLScraperProvider | | public URLScraperProvider()(Code) | | Default constructor.
|
| getFile | | protected File getFile(String pathname)(Code) | | This method is called by getContent() if the url
returned by getURL() is a file url.
Parameters:
pathName - Fully qualified path name of file File Object specified by the pathName or nullif the file does not exists or cannot be read. |
| getHttpContent | | protected StringBuffer getHttpContent(HttpServletRequest req, HttpServletResponse res, String url) throws InterruptedException, MalformedURLException, ProviderException(Code) | | Get the provider's content by retrieving content from the specified
http or https URL.
This method does not handle file URLs. It only handles http or https urls.
The content scraped from the specified url is rewritten if a rewriter is
available using the ruleset returned by getRuleSetID()
This method throws exceptions for certain exceptional conditions instead
of returning an error message in the returned StringBuffer
Parameters:
req - An HttpServletRequest that contains information relatedto this request for content.
Parameters:
res - An HttpServletResponse that allows the provider toinfluence the overall response for the desktop page(besides generating the content).
Parameters:
url - http or https url string Scraped content
exception:
InterruptedException - if there is a timeout whiletrying to get the scraped content
exception:
MalformedURLException - if the url passed in is not a validhttp or https url.
exception:
ProviderException - if there was an error generating thecontent
See Also: com.sun.portal.providers.ProviderException
See Also: URLScraperProvider.getRuleSetID |
| getHttpContent | | protected StringBuffer getHttpContent(HttpServletRequest req, HttpServletResponse res, String url, boolean ubt) throws InterruptedException, MalformedURLException, ProviderException(Code) | | Get the provider's content by retrieving content from the specified
http or https URL.
This method does not handle file URLs. It only handles http or https urls.
The content scraped from the specified url is rewritten if a rewriter is
available using the ruleset returned by getRuleSetID()
This method throws exceptions for certain exceptional conditions instead
of returning an error message in the returned StringBuffer
Parameters:
req - An HttpServletRequest that contains information relatedto this request for content.
Parameters:
res - An HttpServletResponse that allows the provider toinfluence the overall response for the desktop page(besides generating the content).
Parameters:
url - http or https url string
Parameters:
ubt - Indicates whether to track links external to portal Scraped content
exception:
InterruptedException - if there is a timeout whiletrying to get the scraped content
exception:
MalformedURLException - if the url passed in is not a validhttp or https url.
exception:
ProviderException - if there was an error generating thecontent
See Also: com.sun.portal.providers.ProviderException
See Also: URLScraperProvider.getRuleSetID |
| isPresentable | | public boolean isPresentable(HttpServletRequest request)(Code) | | Determines presentability for channels based on this provider.
This overrides the base class's implementation to returns true for all device
Parameters:
request - the HttpServletRequest boolean true for all devices |
| Methods inherited from com.sun.portal.providers.ProfileProviderAdapter | public boolean existsBooleanProperty(String name) throws ProviderException(Code)(Java Doc)
public boolean existsIntegerProperty(String name) throws ProviderException(Code)(Java Doc)
public boolean existsListProperty(String name) throws ProviderException(Code)(Java Doc)
public boolean existsListProperty(String name, boolean localized) throws ProviderException(Code)(Java Doc)
public boolean existsStringProperty(String name) throws ProviderException(Code)(Java Doc)
public boolean existsStringProperty(String name, boolean localized) throws ProviderException(Code)(Java Doc)
public boolean getBooleanProperty(String key) throws ProviderException(Code)(Java Doc)
public boolean getBooleanProperty(String key, List pflist) throws ProviderException(Code)(Java Doc)
public boolean getBooleanProperty(String key, boolean def) throws ProviderException(Code)(Java Doc)
public boolean getBooleanProperty(String key, boolean def, List pflist) throws ProviderException(Code)(Java Doc)
public String getClientProperty(String name)(Code)(Java Doc)
public int getIntegerProperty(String key) throws ProviderException(Code)(Java Doc)
public int getIntegerProperty(String key, List pflist) throws ProviderException(Code)(Java Doc)
public int getIntegerProperty(String key, int def) throws ProviderException(Code)(Java Doc)
public int getIntegerProperty(String key, int def, List pflist) throws ProviderException(Code)(Java Doc)
public List getListProperty(String key) throws ProviderException(Code)(Java Doc)
public List getListProperty(String key, List def) throws ProviderException(Code)(Java Doc)
public Map getMapProperty(String key) throws ProviderException(Code)(Java Doc)
public Map getMapProperty(String key, List pflist) throws ProviderException(Code)(Java Doc)
public Map getMapProperty(String key, boolean localized) throws ProviderException(Code)(Java Doc)
public Map getMapProperty(String key, Map def) throws ProviderException(Code)(Java Doc)
public Map getMapProperty(String key, Map def, List pflist) throws ProviderException(Code)(Java Doc)
public Map getMapProperty(String key, Map def, boolean localized) throws ProviderException(Code)(Java Doc)
public String getStringAttribute(String name)(Code)(Java Doc)
public String getStringProperty(String key) throws ProviderException(Code)(Java Doc)
public String getStringProperty(String key, String def) throws ProviderException(Code)(Java Doc)
public String getStringProperty(String key, List pflist) throws ProviderException(Code)(Java Doc)
public String getStringProperty(String key, String def, List pflist) throws ProviderException(Code)(Java Doc)
public String getStringProperty(String key, boolean localized) throws ProviderException(Code)(Java Doc)
public String getStringProperty(String key, String def, boolean localized) throws ProviderException(Code)(Java Doc)
public StringBuffer getTemplate(String file) throws ProviderException(Code)(Java Doc)
public StringBuffer getTemplate(String file, Hashtable table) throws ProviderException(Code)(Java Doc)
public File getTemplatePath(String file) throws ProviderException(Code)(Java Doc)
public boolean isAllowed(String priv) throws ProviderException(Code)(Java Doc)
public boolean setBooleanProperty(String key, boolean val) throws ProviderException(Code)(Java Doc)
public void setClientProperty(String name, String val)(Code)(Java Doc)
public int setIntegerProperty(String key, int val) throws ProviderException(Code)(Java Doc)
public List setListProperty(String key, List val) throws ProviderException(Code)(Java Doc)
public Map setMapProperty(String key, Map val) throws ProviderException(Code)(Java Doc)
public void setStringAttribute(String name, String val)(Code)(Java Doc)
public String setStringProperty(String key, String val) throws ProviderException(Code)(Java Doc)
|
|
|