Communicating with servers and downloading HTML

Iketh

Active member
Joined
Nov 23, 2009
Messages
40
Programming Experience
Beginner
ok here goes...

The following is my simple block of code to access World of Warcraft's Armory. I want it to access the character named "Ikith" on the server "Bleeding Hollow" and for now print all HTML into a text file.

VB.NET:
    Private Sub btnLoadCharArmory_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnLoadCharArmory.Click

        Dim sLog As String = My.Application.Info.DirectoryPath & "\armorytest.txt"
        Dim oLog As New StreamWriter(sLog)

        Dim request As WebRequest = WebRequest.Create("http://www.wowarmory.com/character-sheet.xml?r=Bleeding+Hollow&n=Ikith")
        Dim response As WebResponse = request.GetResponse()
        Dim reader As StreamReader = New StreamReader(response.GetResponseStream())
        Dim str As String = reader.ReadLine()

        Do While str.Length > 0
            str = reader.ReadLine()
            oLog.WriteLine(str)
        Loop

        oLog.Close()

    End Sub

Instead, it spits out the following, never getting close to the information I need.

HTML:
<html>
<head>
<link href="/favicon.ico" rel="shortcut icon">
<title>The World of Warcraft Armory</title>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
<meta content="The World of Warcraft Armory is a vast searchable database of information for World of Warcraft - taken straight from the real servers. It is the most comprehensive and up-to-date database on the characters, arena teams, guilds, items, and faction rewards of World of Warcraft in existence." name="description">
<style media="screen, projection" type="text/css">
		@import "/_css/master.css";
		@import "/shared/global/menu/topnav/topnav.css";
		
			@import "/_css/int.css";
		
		@import "/_css/_lang/en_us/language.css";
		@import "/_css/_region/us/region.css";
	</style>
<script src="/shared/global/third-party/jquery/jquery.js" type="text/javascript"></script><script src="/shared/global/third-party/jquery/datefunctions.js" type="text/javascript"></script><script src="/shared/global/third-party/jquery/jquery.datepicker.js" type="text/javascript"></script><script src="/shared/global/third-party/jquery/jquery.tablesorter.min.js" type="text/javascript"></script><script src="/shared/global/third-party/jquery/jquery.tablesorter.pager.js" type="text/javascript"></script><script src="/shared/global/third-party/jquery/jquery.bgiframe.min.js" type="text/javascript"></script><script src="/shared/global/third-party/sarissa/0.9.9.3/sarissa.js" type="text/javascript"></script><script src="/_js/common.js" type="text/javascript"></script><script src="/_js/armory.js" type="text/javascript"></script><script type="text/javascript">
		/*  */
		if(Browser.iphone && Number(getcookie2("mobIntPageVisits")) < 3 && getcookie2("hasSeenMobInt") == ""){
			setcookie("mobIntPageOrigin",window.location.href,"session");
			window.location.href = "/mobile-armory-splash.xml";
		}
		/* */
	</script><script type="text/javascript">
		//browser detection		
		if($.browser.msie){
			if($.browser.version == "7.0")		addStylesheet('/_css/browser/ie7.css');
			if($.browser.version == "6.0")		addStylesheet('/_css/browser/ie.css');
		}else if($.browser.mozilla){
			if(parseFloat($.browser.version) <= 1.9)	addStylesheet('/_css/browser/firefox2.css');
		}else if($.browser.opera)				addStylesheet('/_css/browser/opera.css');
		else if($.browser.safari)				addStylesheet('/_css/browser/safari.css');

Clearly I need to communicate some browser information with the server first before the requested information is handed to me. So I did some searching and found that I need to relay my language preference as "English" and the browser info needs to be formatted like "'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2". (Mozilla is the only example I could find. How could I alter this to view safari's or ie7's?)

So how do I communicate this to the server with my web request? If it matters, I'm in a critical time crunch for Monday that has a LOT to do with the success of the program lol.
 
Last edited:
Instead, it spits out the following, never getting close to the information I need.
The html returned is the same as the source when viewed in browser. The problem is that you stop reading when you hit a zero-length text line, which in this source happens at line 33 of 1875 lines. There are several ways to fix your code, you can read until line Is Nothing, which is different than an empty string and means that end of stream is reached. You can also use the StreamReader.ReadToEnd method instead. Simpler is to create a WebClient and use the DownloadFile/DownloadString method, just two lines of code. My.Computer.Network.DownloadFile is also an option.
 
VB.NET:
    Private Sub btnLoadCharArmory_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles btnLoadCharArmory.Click

        Dim sLog As String = My.Application.Info.DirectoryPath & "\armorytest.txt"
        If File.Exists(sLog) Then File.Delete(sLog)
        My.Computer.Network.DownloadFile("http://www.wowarmory.com/character-sheet.xml?r=Bleeding+Hollow&n=Ikith", sLog)

    End Sub

Thanks, this code gives me what I need. I just didn't want to work with files if I didn't need to. Not sure what the webclient is for that you stated, i'm a total noob using vb with networks.
 
Thanks a lot for the help so far John, but I have a new problem.

At the URL The World of Warcraft Armory, the information I need is actually not in that HTML code. It's not downloaded until you hover the mouse over a piece of gear...

I need all gear information the character is wearing, so how do I invoke those mouseover handles and/or find the URL for downloading each piece of gear? I combed through the 1875 lines of HTML and the only information on the items in that code is the item's ID number (beginning on line 879 with the helm), which I can use to download the item's information, but that method won't include the player's additions to the item (enchants and gems).

I need to download the tool-tip window's information when you mouseover a piece of gear.
 
Last edited:
The address you changed to is a Xml file, it is formatted for display by the referenced xsl stylesheet, this is what sets up the rendered html, the xml is data only. For this document you have no other option than to read the xsl to find out which other document is loaded and download that. Most likely you'll end up somewhere in a Javascript before you know what exactly is going on.
 
Here is all the html for the gear on the page, but I don't see an xsl reference that you speak of. It's calling a class named "staticTip itemToolTip gearFrame"... if that's the xsl, how do I interact with it?

VB.NET:
874 <div class="profile-master2">

875 <div class="leftItems">

876 <div class="gearItem" style="

877 					background-image: url('/wow-icons/_images/51x51/inv_helmet_102.jpg')

878 				">

879 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=47771" id="i=47771&r=Bleeding+Hollow&n=Ikith&s=0">

880 <div class="upgradeBox"></div>

881 </a>

882 <div class="fly-horz">

883 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=47771">Find an Upgrade</a>

884 </div>

885 </div>

886 <div class="gearItem" style="

887 					background-image: url('/wow-icons/_images/51x51/inv_jewelry_necklace_41.jpg')

888 				">

889 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=45822" id="i=45822&r=Bleeding+Hollow&n=Ikith&s=1">

890 <div class="upgradeBox"></div>

891 </a>

892 <div class="fly-horz">

893 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=45822">Find an Upgrade</a>

894 </div>

895 </div>

896 <div class="gearItem" style="

897 					background-image: url('/wow-icons/_images/51x51/inv_shoulder_35.jpg')

898 				">

899 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=47768" id="i=47768&r=Bleeding+Hollow&n=Ikith&s=2">

900 <div class="upgradeBox"></div>

901 </a>

902 <div class="fly-horz">

903 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=47768">Find an Upgrade</a>

904 </div>

905 </div>

906 <div class="gearItem" style="

907 					background-image: url('/wow-icons/_images/51x51/inv_misc_cape_18.jpg')

908 				">

909 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=47256" id="i=47256&r=Bleeding+Hollow&n=Ikith&s=14">

910 <div class="upgradeBox"></div>

911 </a>

912 <div class="fly-horz">

913 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=47256">Find an Upgrade</a>

914 </div>

915 </div>

916 <div class="gearItem" style="

917 					background-image: url('/wow-icons/_images/51x51/inv_chest_cloth_73.jpg')

918 				">

919 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=47264" id="i=47264&r=Bleeding+Hollow&n=Ikith&s=4">

920 <div class="upgradeBox"></div>

921 </a>

922 <div class="fly-horz">

923 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=47264">Find an Upgrade</a>

924 </div>

925 </div>

926 <div class="gearItem">

927 <a class="staticTip noUpgrade gearFrame" href="/item-info.xml?i=" id="i=&r=Bleeding+Hollow&n=Ikith&s=3" onmouseover="setTipText(getEmptySlotToolTip('3','8'))"></a>

928 </div>

929 <div class="gearItem">

930 <a class="staticTip noUpgrade gearFrame" href="/item-info.xml?i=" id="i=&r=Bleeding+Hollow&n=Ikith&s=18" onmouseover="setTipText(getEmptySlotToolTip('18','8'))"></a>

931 </div>

932 <div class="gearItem" style="

933 					background-image: url('/wow-icons/_images/51x51/inv_bracer_32a.jpg')

934 				">

935 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=47861" id="i=47861&r=Bleeding+Hollow&n=Ikith&s=8">

936 <div class="upgradeBox"></div>

937 </a>

938 <div class="fly-horz">

939 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=47861">Find an Upgrade</a>

940 </div>

941 </div>

942 </div>

943 <div class="rightItems">

944 <div class="gearItem" style="

945 					background-image: url('/wow-icons/_images/51x51/inv_gauntlets_56.jpg')

946 				">

947 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=47772" id="i=47772&r=Bleeding+Hollow&n=Ikith&s=9">

948 <div class="upgradeBox"></div>

949 </a>

950 <div class="fly-horz">

951 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=47772">Find an Upgrade</a>

952 </div>

953 </div>

954 <div class="gearItem" style="

955 					background-image: url('/wow-icons/_images/51x51/inv_belt_52.jpg')

956 				">

957 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=47994" id="i=47994&r=Bleeding+Hollow&n=Ikith&s=5">

958 <div class="upgradeBox"></div>

959 </a>

960 <div class="fly-horz">

961 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=47994">Find an Upgrade</a>

962 </div>

963 </div>

964 <div class="gearItem" style="

965 					background-image: url('/wow-icons/_images/51x51/inv_pants_cloth_20.jpg')

966 				">

967 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=47770" id="i=47770&r=Bleeding+Hollow&n=Ikith&s=6">

968 <div class="upgradeBox"></div>

969 </a>

970 <div class="fly-horz">

971 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=47770">Find an Upgrade</a>

972 </div>

973 </div>

974 <div class="gearItem" style="

975 					background-image: url('/wow-icons/_images/51x51/inv_boots_cloth_19.jpg')

976 				">

977 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=45135" id="i=45135&r=Bleeding+Hollow&n=Ikith&s=7">

978 <div class="upgradeBox"></div>

979 </a>

980 <div class="fly-horz">

981 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=45135">Find an Upgrade</a>

982 </div>

983 </div>

984 <div class="gearItem" style="

985 					background-image: url('/wow-icons/_images/51x51/inv_jewelry_ring_68.jpg')

986 				">

987 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=40719" id="i=40719&r=Bleeding+Hollow&n=Ikith&s=10">

988 <div class="upgradeBox"></div>

989 </a>

990 <div class="fly-horz">

991 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=40719">Find an Upgrade</a>

992 </div>

993 </div>

994 <div class="gearItem" style="

995 					background-image: url('/wow-icons/_images/51x51/inv_jewelry_ring_54.jpg')

996 				">

997 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=47732" id="i=47732&r=Bleeding+Hollow&n=Ikith&s=11">

998 <div class="upgradeBox"></div>

999 </a>

1000 <div class="fly-horz">

1001 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=47732">Find an Upgrade</a>

1002 </div>

1003 </div>

1004 <div class="gearItem" style="

1005 					background-image: url('/wow-icons/_images/51x51/inv_offhand_hyjal_d_01.jpg')

1006 				">

1007 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=40432" id="i=40432&r=Bleeding+Hollow&n=Ikith&s=12">

1008 <div class="upgradeBox"></div>

1009 </a>

1010 <div class="fly-horz">

1011 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=40432">Find an Upgrade</a>

1012 </div>

1013 </div>

1014 <div class="gearItem" style="

1015 					background-image: url('/wow-icons/_images/51x51/inv_misc_gem_bloodstone_03.jpg')

1016 				">

1017 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=48724" id="i=48724&r=Bleeding+Hollow&n=Ikith&s=13">

1018 <div class="upgradeBox"></div>

1019 </a>

1020 <div class="fly-horz">

1021 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=48724">Find an Upgrade</a>

1022 </div>

1023 </div>

1024 </div>

1025 <div class="bottomItems">

1026 <div class="gearItem" style="

1027 					background-image: url('/wow-icons/_images/51x51/inv_weapon_shortblade_93.jpg')

1028 				">

1029 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=47261" id="i=47261&r=Bleeding+Hollow&n=Ikith&s=15">

1030 <div class="upgradeBox"></div>

1031 </a>

1032 <div class="fly-horz">

1033 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=47261">Find an Upgrade</a>

1034 </div>

1035 </div>

1036 <div class="gearItem" style="

1037 					background-image: url('/wow-icons/_images/51x51/inv_offhand_stratholme_a_02.jpg')

1038 				">

1039 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=48032" id="i=48032&r=Bleeding+Hollow&n=Ikith&s=16">

1040 <div class="upgradeBox"></div>

1041 </a>

1042 <div class="fly-horz">

1043 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=48032">Find an Upgrade</a>

1044 </div>

1045 </div>

1046 <div class="gearItem" style="

1047 					background-image: url('/wow-icons/_images/51x51/inv_wand_16.jpg')

1048 				">

1049 <a class="staticTip itemToolTip gearFrame" href="/item-info.xml?i=39712" id="i=39712&r=Bleeding+Hollow&n=Ikith&s=17">

1050 <div class="upgradeBox"></div>

1051 </a>

1052 <div class="fly-horz">

1053 <a class="upgrd" href="/search.xml?searchType=items&pr=Bleeding+Hollow&pn=Ikith&pi=39712">Find an Upgrade</a>

1054 </div>

1055 </div>

1056 </div>
 
Last edited:
The new url you posted point to an xml file, not a html document. It is not the same url as I answered to in post 2. Xml is usually easiest handled with the XmlDocument class, but the dynamic content rendered with the xsl stylesheet and javascript it is a different matter, you just have to figure out their client side code to find out where that data comes from.
 
I don't understand... I didn't alter the url at all... in fact I just made sure they are the same address...
 
Just to clarify, all I want to do is screen scrape the information that downloads, I don't care what format it's in.

Here is a function that queries the data I'm looking for.

VB.NET:
function armory_get($region, $file, $query) {
        global $url_armory;

        $msg = http_get($url_armory[$region] . "$file.xml?$query", array('redirect' => 10,'useragent' => 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2'), $info);

        // Second attempt on DNS errors
        if (isset($info) && !empty($info['error']) && $info['error'] == "name lookup timed out") $msg = http_get($url_armory[$region] . "$file.xml?$query", array('redirect' => 10,'useragent' => 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.2) Gecko/20070219 Firefox/2.0.0.2'), $info);

        if (isset($info) && !empty($info['error'])) throw new APBException("Failed to retrieve profile, error \"$info[error]\"", APB_EX_SKIP);
        if (isset($info) && isset($info['response_code']) && $info['response_code'] != 200) throw new APBException("Failed to retrieve profile, HTTP code $info[response_code]", APB_EX_SKIP);
        if (empty($msg)) throw new APBException("Failed to retrieve profile, unknown error", APB_EX_SKIP);

        return http_parse_message($msg)->body;
}

And here is code that calls the function... problem is, I don't even know what language this is lol... I want to do what this little program is doing in VB, but I don't know where to begin learning this. I don't even know what to google for, and that's pretty bad.

VB.NET:
        foreach ($names as $rawname) {
                // encode before splitting so that it only has to be encoded once
                $name = urlencode(toUTF8($rawname));

                if (-1 == ($pos = strpos($name, "+"))) {
                        trigger_error($msg = "Incorrectly formatted line \"$rawname\".", E_USER_WARNING);
                        $table .= "<!-- $msg -->\n";
                        if (count($skipped)) array_push($names, array_shift($skipped));
                        continue;
                }

                $region = strtoupper(substr($name, 0, $pos));
                $name = substr($name, $pos + 1);

                if (!$url_armory[$region]) {
                        trigger_error($msg = "Unrecognized region: \"$region\".", E_USER_WARNING);
                        $table .= "<!-- $msg -->\n";
                        if (count($skipped)) array_push($names, array_shift($skipped));
                        continue;
                }

                if (-1 == ($pos = strrpos($name, "+"))) {
                        trigger_error($msg = "Incorrectly formatted line \"$rawname\".", E_USER_WARNING);
                        $table .= "<!-- $msg -->\n";
                        if (count($skipped)) array_push($names, array_shift($skipped));
                        continue;
                }

                $realm = substr($name, 0, $pos);
                $name = ucfirst(strtolower(substr($name, $pos + 1)));

                if (!isset($done_chars[$region])) $done_chars[$region] = array();
                if (!isset($done_chars[$region][$realm])) $done_chars[$region][$realm] = array();

                if (isset($done_chars[$region][$realm][$name])) {
                        $char = $done_chars[$region][$realm][$name];
                } else {
                        $char = array("region" => $region);

                        foreach ($files as $file => $fields) {
                                try {
                                        if ($verbose) echo "Retrieving profile for \"$region $realm $name\"...\n";
                                        if (!($xml = armory_get($region, $file, "r=$realm&n=$name"))) {
                                                trigger_error($msg = "Could not read profile for character \"$name\" on server \"$realm\" in region \"$region\".", E_USER_WARNING);
                                                $table .= "<!-- $msg -->\n";
                                                if (count($skipped)) array_push($names, array_shift($skipped));
                                                continue 2;
                                        }
                                } catch (APBException $ex) {
                                        if ($ex->getCode() == APB_EX_FATAL) trigger_error($ex, E_USER_ERROR);
                                        trigger_error($ex, E_USER_WARNING);
                                        if (count($skipped)) array_push($names, array_shift($skipped));
                                        continue 2;
                                } catch (Exception $ex) {
                                        // Don't expose unknown exceptions to the Wiki results.
                                        trigger_error(($msg = "Unexpected exception retrieving profile data for character \"$name\" on server \"$realm\" in region \"$region\".") . ": $ex", E_USER_WARNING);
                                        $table .= "<!-- $msg. -->\n";
                                        if (count($skipped)) array_push($names, array_shift($skipped));
                                        continue 2;
                                }

                                if (false !== strpos($xml, "<characterInfo/>")) {
                                        trigger_error($msg = "No character named \"$name\" on server \"$realm\" in region \"$region\" or other Armory error.", E_USER_WARNING);
                                        $table .= "<!-- $msg -->\n";
                                        if (count($skipped)) array_push($names, array_shift($skipped));
                                        continue 2;
                                }

                                try {
                                        $dom = new DOMDocument();
                                        $dom->loadXML($xml);
                                        $xpath = new DOMXPath($dom);
                                } catch (Exception $ex) {
                                        // Don't expose unknown exceptions to the Wiki results.
                                        trigger_error(($msg = "Error parsing profile data for character \"$name\" on server \"$realm\" in region \"$region\".") . ": $ex", E_USER_WARNING);
                                        $table .= "<!-- $msg -->\n";
                                        if (count($skipped)) array_push($names, array_shift($skipped));
                                        continue 2;
                                }

                                foreach ($fields as $key => $path) {
                                        $nodes = $xpath->query($path);
                                        if ($nodes && $nodes->length) $char[$key] = $nodes->item(0)->value;
                                        else $char[$key] = "";
                                }
                        }

                        // The class name is required to even set the row template; without it, give up.
                        if (!$char["class"]) {
                                trigger_error($msg = "Could not determine class for \"$region $realm $name\".", E_USER_WARNING);
                                $table .= "<!-- $msg -->\n";
                                if (count($skipped)) array_push($names, array_shift($skipped));
                                continue;
                        }

                        // "guild" should be the only field which can contain <, >, &, or | (all of which are dangerous)
                        if ($char["guild"]) $char["guild"] = str_replace("|", "|", htmlspecialchars($char["guild"], ENT_NOQUOTES));

                        foreach (array("str", "agi", "sta", "int", "spi", "arm") as $key) {
                                if ($char[$key] == $char["${key}base"]) {
                                        $char["${key}base"] = "";
                                        $char["${key}plus"] = "";
                                } else {
                                        $char["${key}plus"] = $char[$key] - $char["${key}base"];
                                }
                        }

                        if (is_numeric($char["talent1"])) {
                                $spec = array($char["talent1"], $char["talent2"], $char["talent3"]);
                                arsort($spec);
                                $spec = array_keys($spec);
                                if ($char["talent" . ($spec[0] + 1)] == 0) {
                                        $char["spec1"] = "None";
                                        $char["spec2"] = "";
                                } else {
                                        $lim = $char["talent" . ($spec[0] + 1)] / 2;
                                        if ($char["talent" . ($spec[2] + 1)] >= $lim) {
                                                $char["spec1"] = "Hybrid";
                                                $char["spec2"] = "";
                                        } else if ($char["talent" . ($spec[1] + 1)] >= $lim) {
                                                $char["spec1"] = $spec[0] + 1;
                                                $char["spec2"] = $spec[1] + 1;
                                        } else {
                                                $char["spec1"] = $spec[0] + 1;
                                                $char["spec2"] = "";
                                        }
                                }

                                $char["talents"] = "$char[talent1]/$char[talent2]/$char[talent3]";
                        } else {
                                $char["spec1"] = "";
                                $char["spec2"] = "";
                                $char["talents"] = "";
                        }

                        $done_chars[$region][$realm][$name] = $char;
                }

                $table .= preg_replace('/\|+}}/', '}}', "{{User:ArmoryProfileBot/Row$char[class]|$char[region]|$char[realm]|$char[name]|$char[gender]|$char[race]|$char[level]|$char[guild]|$char[spec1]|$char[spec2]|$char[talents]|$char[health]|$char[mana]|$char[str]|$char[strbase]|$char[strplus]|$char[agi]|$char[agibase]|$char[agiplus]|$char[sta]|$char[stabase]|$char[staplus]|$char[int]|$char[intbase]|$char[intplus]|$char[spi]|$char[spibase]|$char[spiplus]|$char[arm]|$char[armbase]|$char[armplus]|$char[prof1]|$char[prof1val]|$char[prof2]|$char[prof2val]|$char[cooking]|$char[firstaid]|$char[fishing]|$char[riding]}}\n");
        }
 
Last edited:
Hmm weird, browser and .Net request return different sources, it's the combination of request headers I guess, open the url in browser and 'view source' and you'll see the xml I mentioned. It would normally be much easier to get data from such xml document than to parse html/string. The same goes for a /item-info.xml?i=47771 url.

All that $//;}{ sure looks like Javascript, where did you find it? In a .js file?
 
I'll add a sample that loads xml and get one attribute, it is not very obvious when that url returns html or xml, but the handling of the data is very different. The request headers here is basically just copied from Firefox, until it returned same xml source as browser.
VB.NET:
Dim url As String = "http://www.wowarmory.com/character-sheet.xml?r=Bleeding+Hollow&n=Ikith"
Dim cli As New Net.WebClient
cli.Headers(Net.HttpRequestHeader.UserAgent) = "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-EN; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729)"
cli.Headers(Net.HttpRequestHeader.Accept) = "application/xml"
cli.Headers(Net.HttpRequestHeader.AcceptEncoding) = "gzip"

Dim doc As New Xml.XmlDocument
Dim comp As New IO.Compression.GZipStream(cli.OpenRead(url), IO.Compression.CompressionMode.Decompress)
doc.Load(comp)
comp.Close()

Dim chr As Xml.XmlNode = doc.SelectSingleNode("//characterInfo/character")
Dim points As String = chr.Attributes("points").Value 'currently returns value 3530
 
it's php... sorry didn't realize that or I would've wrapped it. Quote from the site: "This bot was written in PHP and uses the HTTP extension." It's outdated now anyhow, more than a year since an update.

Here is another program. It's "Flex and PHP"? It's a widget and really small, someone's gotta see how it's grabbing the info... Meanwhile I'm gonna play around with your sample, thanks a lot john.

PHP:
<?xml version="1.0" encoding="utf-8"?>
<mx:Application 
 xmlns:mx="http://www.adobe.com/2006/mxml" 
 layout="absolute"
 width="250" height="300">
  <mx:Script>
   <![CDATA[
     import com.adobe.viewsource.ViewSource;
     import mx.controls.dataGridClasses.DataGridColumn;
     import mx.messaging.messages.ErrorMessage;
     import mx.collections.ArrayCollection;
     import mx.controls.Alert;
     import mx.rpc.events.FaultEvent;
     import mx.rpc.events.ResultEvent;
     
     [Bindable]
     private var term:String;
     
     [Bindable]
     private var items:ArrayCollection;
     
     private function search():void
     {
       wowsearch.send();
     }
     
     private function handleResult(e:ResultEvent):void
     {
       items = e.result.page.armorySearch.searchResults.items.item;
     }
     
     private function handleFault(e:FaultEvent):void
     {
       Alert.show((e.message as ErrorMessage).faultString);
     }
     
     private function displayLevel(item:Object, col:DataGridColumn):String
     {
       if(item) {
         for each(var o:Object in item.filter) {
           if(o.name == "itemLevel") {
             return o.value.toString();
           }
         }  
       }
       
       return "";
     }
   ]]>
 </mx:Script>
  <mx:Binding source="searchTerm.text" destination="term" />
  <mx:HTTPService
   id="wowsearch" 
   url="/sites/default/phpExamples/576/wowarmory.php" 
   method="GET" resultFormat="object" 
   result="handleResult(event)" fault="handleFault(event)"
   showBusyCursor="true">
    <mx:request>
      <term>{term}</term>
    </mx:request>
  </mx:HTTPService>
  <mx:Label x="5" y="4" text="Search WoW Items" 
   fontSize="14" fontWeight="bold"/>
  <mx:TextInput x="5" y="31" width="161" id="searchTerm" enter="search()"/>
  <mx:Button x="175" y="31" label="Search" width="70" click="search()"/>
  <mx:DataGrid x="5" y="61" width="240" height="235" 
   dataProvider="{items}" rowHeight="25" fontSize="9">
    <mx:columns>
      <mx:DataGridColumn headerText="" itemRenderer="ItemImage"
       width="25"/>
      <mx:DataGridColumn headerText="Item Name" itemRenderer="ItemName"/>
      <mx:DataGridColumn headerText="Level" labelFunction="displayLevel" 
       width="45"/>
    </mx:columns>
  </mx:DataGrid>
</mx:Application>
 
Back
Top