[Home] [Downloads] [Search] [Help/forum]

Register forum user name Search FAQ

Gammon Forum

[Folder]  Entire forum
-> [Folder]  MUSHclient
. -> [Folder]  Lua
. . -> [Subject]  can't download dropbox public links
can't download dropbox public links

It is now over 60 days since the last post. This thread is closed.     [Refresh] Refresh page

Pages: 1 2  

Posted by Victorious   (89 posts)  [Biography] bio
Date Thu 24 Jul 2014 03:12 PM (UTC)
Message
I've never had a problem downloading dropbox public links with lua before, until recently. The url I'm trying to download is https://dl.dropboxusercontent.com/u/2138237/version.txt

I'm not having any problems downloading stuff from other websites. Whenever I try downloading a dropbox public link, it returns a http 302 error for some reason. I've also tried modifying the link, following advice in dropbox's help centre like adding ?dl=1 to the end of the url, but I've not had any luck.

Is anyone able to download any dropbox public links with lua? I'm inclined to think that this is something that is caused by a dropbox change, as I've not made any changes to my code for a long time.
[Go to top] top
Posted by Fiendish   USA  (2,514 posts)  [Biography] bio   Global Moderator
Date Reply #1 on Thu 24 Jul 2014 03:32 PM (UTC)

Amended on Mon 06 Nov 2017 05:38 PM (UTC) by Fiendish

Message
LuaSocket can't do HTTPS and instead just silently shunts you to HTTP, which will not work on the modern secure web.
You will have to also wrap your connection with LuaSec.

Quote:

$ curl -v http://dl.dropboxusercontent.com/u/2138237/version.txt

* About to connect() to dl.dropboxusercontent.com port 80 (#0)
* Trying 107.21.220.74...
* Connected to dl.dropboxusercontent.com (107.21.220.74) port 80 (#0)
> GET /u/2138237/version.txt HTTP/1.1
> User-Agent: curl/7.29.0
> Host: dl.dropboxusercontent.com
> Accept: */*
>
< HTTP/1.1 302 FOUND
< cache-control: no-cache
< Content-Type: text/html; charset=utf-8
< Date: Thu, 24 Jul 2014 16:33:48 GMT
< location: https://dl.dropboxusercontent.com/u/2138237/version.txt
< pragma: no-cache
< Server: nginx
< set-cookie: flash=; Domain=dropbox.com; expires=Thu, 24 Jul 2014 16:33:48 GMT; Path=/; httponly
< set-cookie: bang=; Domain=dropbox.com; expires=Thu, 24 Jul 2014 16:33:48 GMT; Path=/; httponly
< set-cookie: uc_session=HKU7emqrjWnamp53RyvEQGYzOQrcBOtViaO9MbI7dVs94MBQO2l8dpoKFMfHCfa1; Domain=dropboxusercontent.com; Path=/; secure; httponly
< Content-Length: 374
< Connection: keep-alive
<
<html>
<head><title>Found</title></head>
<body>
<h1>Found</h1>
<p>The resource was found at <a href="https://dl.dropboxusercontent.com/u/2138237/version.txt">https://dl.dropboxusercontent.com/u/2138237/version.txt</a>;
you should be redirected automatically.

<!-- --></p>
<hr noshade>
<div align="right">WSGI Server</div>
</body>
</html>
* Connection #0 to host dl.dropboxusercontent.com left intact


I ran into the same thing when I started using github, which also does not allow HTTP connections.

I now have a wrapper for doing asynchronous HTTPS using LuaSec, LuaSocket, and lua-llthreads.

Relevant files in my repo are (I think this is all of them):
https://github.com/fiendish/aardwolfclientpackage/blob/MUSHclient/MUSHclient/lua/async.lua
https://github.com/fiendish/aardwolfclientpackage/blob/MUSHclient/MUSHclient/lua/ssl/https.lua
https://github.com/fiendish/aardwolfclientpackage/blob/MUSHclient/MUSHclient/lua/ssl.lua
https://github.com/fiendish/aardwolfclientpackage/blob/development/MUSHclient/llthreads.dll
[EDIT] I've since switched from LuaSec to Lua-openssl because luasec was not thread-safe and the lua-openssl creator actually pays attention to pull requests and comments.
https://github.com/zhaozg/lua-openssl

The complete set of files related to sockets, ssl, and threading should be:

MUSHclient/socket/*
MUSHclient/lua/ssl/*
MUSHclient/mime/*
MUSHclient/lua/async.lua (for asynchronous background sockets)
MUSHclient/lua/ssl.lua
MUSHclient/lua/socket.lua
MUSHclient/libeay32.dll
MUSHclient/llthreads.dll (for asynchronous background sockets)
MUSHclient/openssl.dll
MUSHclient/ssleay32.dll

You might also need to replace your lua dll with mine (there are two), since I use LuaJIT instead of PUC-Rio Lua, and then also include the msvcr100 dll if you don't have it in your system32 directory.

As Nick mentions below, Dependency Walker is a great tool for determining what's still missing.


Then...

async_ok, async = pcall (require, "async")
thread = nil


Then somewhere where you want to initiate the background request...

if async_ok and not thread then
   thread = async.request(version_url, "HTTPS")
end


Then at some later point when you expect the request to be complete...

retval, page, status, headers, full_status = thread:join()
thread = nil
if status == 200 then -- 200 is the HTTP status code for "OK"
   -- parse page
else
   -- access error
end


The async stuff isn't necessary, it just prevents long pauses on slow server response ( you can test with http://fake-response.appspot.com/?sleep=5 ) because the version of LuaSec I'm using didn't seem to support it natively.
https://github.com/fiendish/aardwolfclientpackage
[Go to top] top
Posted by Victorious   (89 posts)  [Biography] bio
Date Reply #2 on Sat 26 Jul 2014 09:35 AM (UTC)

Amended on Sat 26 Jul 2014 11:14 AM (UTC) by Victorious

Message
error loading module 'llthreads' from file '.\llthreads.dll':
The specified procedure could not be found.

Edit: I've also downloaded the aardwolf package update checker and installed it into my mushclient installation. I think its also unable to load the dll as I get the following error:

Aardwolf MUSHclient Package Update Check Error:
Failed to load plugin network component.

Edit2: it looks like your aardwolf package is using a different lua 5.1.dll. Trying to substitute the standard lua.dll with the one from the package causes a lot of my lua code that is valid to break.
[Go to top] top
Posted by Fiendish   USA  (2,514 posts)  [Biography] bio   Global Moderator
Date Reply #3 on Sat 26 Jul 2014 06:42 PM (UTC)

Amended on Sat 26 Jul 2014 06:49 PM (UTC) by Fiendish

Message
It's possible that llthreads requires luajit. I don't remember, though I see no reason why it would. You would have to check the github repository for that project to find out.

The luajit dll that I distribute is fully compatible with the Lua 5.1 dll distributed with MUSHclient. If functions aren't working then you either only copied the stub dll and not the real one (I believe you must take both either for luasec or llthreads because one of them uses the alternate canonical Lua dll name that MUSHclient.exe does not use) or you didn't take the vc100 redistributable dll which is also required due to compiler differences.
https://github.com/fiendish/aardwolfclientpackage
[Go to top] top
Posted by Nick Gammon   Australia  (22,975 posts)  [Biography] bio   Forum Administrator
Date Reply #4 on Sat 26 Jul 2014 09:23 PM (UTC)
Message
You can use a/the dependency walker to find exactly which files (DLLs) are missing.


http://www.dependencywalker.com/

Something like that came with Visual C++ and I used to use that in this sort of situation. It shows if there is a missing DLL, and sometimes it can be something not obvious, like one of the Microsoft library DLLs.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top
Posted by Nick Gammon   Australia  (22,975 posts)  [Biography] bio   Forum Administrator
Date Reply #5 on Sat 26 Jul 2014 09:38 PM (UTC)
Message
From my copy of Aardwolf client, here are the dependencies of Lua5.1.dll:

- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top
Posted by Victorious   (89 posts)  [Biography] bio
Date Reply #6 on Sun 27 Jul 2014 06:25 AM (UTC)

Amended on Sun 27 Jul 2014 07:24 AM (UTC) by Victorious

Message
Hm, it seems that this version of the lua dlls have problems with the \ character. For instance,
f = io.input (GetInfo(56).."\worlds/ctips.txt")

needs to be changed to
f = io.input (GetInfo(56).."/worlds/ctips.txt")

another one:
local ret = os.execute("cd ..\.. & update_lite.exe")
needs to be
local ret = os.execute("cd ../.. & update_lite.exe")

Is it possible to make it work properly with the without having to replace the \ characters?

Edit: I've managed to get the dll to load, but it produces a error creating context when I try to get the results of the request. It works properly when I use the aardwolf client package though, so maybe I may have missed a step in the installation.

retval, page, status, headers, full_status = thread:join()
thread = nil
if status == 200 then -- 200 is the HTTP status code for "OK"
print("status "..status)
else
print("status is "..status)
end

I copied lua 5.1.dll, lua 51.dll, ssl.dll, async.lua, https.lua , ssl.lua and llthreads.dll from the aardwolf package. I didn't copy msvcr100.dll as it seems to affect screen reader's ability to work with mushclient; removing that from the aardwolf package makes it accessible, without breaking the ability to use the code above. I can't think of any steps that I missed.
[Go to top] top
Posted by Nick Gammon   Australia  (22,975 posts)  [Biography] bio   Forum Administrator
Date Reply #7 on Sun 27 Jul 2014 08:04 AM (UTC)
Message
Quote:

For instance,
f = io.input (GetInfo(56).."\worlds/ctips.txt")


In Lua, if you want a backslash inside a string you need to double it:


  f = io.input (GetInfo(56).."\\worlds/ctips.txt")


But do you really have a "worlds" folder in your root directory?
- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top
Posted by Victorious   (89 posts)  [Biography] bio
Date Reply #8 on Sun 27 Jul 2014 08:59 AM (UTC)

Amended on Sun 27 Jul 2014 01:27 PM (UTC) by Victorious

Message
Just did some research on lua jit, and it looks like lua jit's just stricter on valid escape sequences.

Edit: I managed to figure out what was wrong, and managed to get everything working.

One (hopefully) last thing, is there a way for you to tell whether the request has been completed? I noticed that in your aardwolf package checker, the main() function is called using do after. I'm hoping this is possible, so that I can use this to implement support for a plugin updater.

Thanks fiendish for sharing this; I can now do https downloads, and it doesn't hang mushclient anymore.
[Go to top] top
Posted by Fiendish   USA  (2,514 posts)  [Biography] bio   Global Moderator
Date Reply #9 on Sun 27 Jul 2014 02:21 PM (UTC)

Amended on Sun 27 Jul 2014 02:26 PM (UTC) by Fiendish

Message
Victorious said:

Just did some research on lua jit, and it looks like lua jit's just stricter on valid escape sequences.

Ah, now that you mention it I do remember also having that conversion speedbump when I switched over. Yeah, LuaJIT uses the Lua 5.2 strictness for invalid escape sequences.

Quote:

One (hopefully) last thing, is there a way for you to tell whether the request has been completed? I noticed that in your aardwolf package checker, the main() function is called using do after. I'm hoping this is possible, so that I can use this to implement support for a plugin updater.

It may be possible, but maybe not with the simple wrapper I made. I think you probably have to use interthread messaging via luasocket or zeromq for that. It's a good idea. I'll look into it later, but if you get it working before I get a chance I'd love to hear.

Quote:

Thanks fiendish for sharing this; I can now do https downloads, and it doesn't hang mushclient anymore.

Cool. :)
https://github.com/fiendish/aardwolfclientpackage
[Go to top] top
Posted by Victorious   (89 posts)  [Biography] bio
Date Reply #10 on Sun 27 Jul 2014 03:27 PM (UTC)
Message
Unfortunately, I'm no where nearly good enough to code something that complex. Look forward to seeing what you come up with :)
[Go to top] top
Posted by Victorious   (89 posts)  [Biography] bio
Date Reply #11 on Sun 27 Jul 2014 03:41 PM (UTC)
Message
Hm, I wonder if this will work? It looks like a simpler solution to what you suggested.
https://github.com/moteus/lua-llthreads2
[Go to top] top
Posted by Fiendish   USA  (2,514 posts)  [Biography] bio   Global Moderator
Date Reply #12 on Wed 30 Jul 2014 04:37 AM (UTC)

Amended on Wed 30 Jul 2014 02:27 PM (UTC) by Fiendish

Message
I think it might.
A new dll is compiled at
https://github.com/fiendish/aardwolfclientpackage/raw/development/MUSHclient/llthreads.dll

It should now let you check if thread:alive()
Let me know if it works for you?

[EDIT] amended original post with new dll location, because I haven't pushed it out of the development branch yet.
https://github.com/fiendish/aardwolfclientpackage
[Go to top] top
Posted by Victorious   (89 posts)  [Biography] bio
Date Reply #13 on Wed 30 Jul 2014 06:43 AM (UTC)
Message
Wow, thanks a lot. It works :)
[Go to top] top
Posted by Fiendish   USA  (2,514 posts)  [Biography] bio   Global Moderator
Date Reply #14 on Wed 30 Jul 2014 02:32 PM (UTC)

Amended on Wed 30 Jul 2014 02:33 PM (UTC) by Fiendish

Message
Quote:
I didn't copy msvcr100.dll as it seems to affect screen reader's ability to work with mushclient; removing that from the aardwolf package makes it accessible, without breaking the ability to use the code above.

Hopefully you're still reading the thread. Can you clear this up for me? My understanding was that msvcr100.dll is necessary for my dlls to work because they are compiled with VC2010. How do they still load for you without it?
https://github.com/fiendish/aardwolfclientpackage
[Go to top] top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.

66,863 views.

This is page 1, subject is 2 pages long: 1 2  [Next page]

It is now over 60 days since the last post. This thread is closed.     [Refresh] Refresh page

Go to topic:           Search the forum

[Go to top] top

Quick links: MUSHclient. MUSHclient help. Forum shortcuts. Posting templates. Lua modules. Lua documentation.

Information and images on this site are licensed under the Creative Commons Attribution 3.0 Australia License unless stated otherwise.

[Home]

Written by Nick Gammon - 5K   profile for Nick Gammon on Stack Exchange, a network of free, community-driven Q&A sites   Marriage equality

Comments to: Gammon Software support
[RH click to get RSS URL] Forum RSS feed ( https://gammon.com.au/rss/forum.xml )

[Best viewed with any browser - 2K]    [Hosted at HostDash]